---
title: "Data Backfilling to Storages"
slug: "data-backfilling-to-storages"
description: "Load historical data from any date range to your storage with Dataddo. Easily load historical data from sources to destinations with one-time requests."
tags: ["Data warehouse", "Database", "Data flow", "Data backfilling"]
updated: 2025-03-23T16:48:03Z
published: 2025-03-23T16:48:03Z
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataddo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Backfilling to Storages

Dataddo's **Data Backfilling** feature enables you to add historical data to [storage](/docs/data-storages) solutions like [BigQuery](/docs/google-bigquery), [Snowflake](/docs/snowflake), or [Databricks](/docs/databricks). This makes your data storage comprehensive, including both current and past data, improving your analytics and reporting capabilities.

          **DATADDO TIP**

          

See [Full Data Re-Sync](/docs/full-data-re-sync) to load **all your historical data at once**.

If your destination is a **dashboarding app** like [Looker Studio](/docs/looker-studio), [Power BI](/docs/power-bi), or [Tableau](/docs/tableau), refer to [**this article**](/docs/data-backfilling-to-dashboarding-apps).

## The Process

1. Go to the [**Sources**](https://app.dataddo.com/sources) page and [trigger the extraction](/docs/data-backfilling-to-storages#trigger-the-extraction) for your selected source and choose your date range.
  1. If the UI does not offer a date range picker, you can temporarily [override the source configuration](/docs/data-backfilling-to-storages#override-source-configuration).
2. Initiate the [data writing](/v1/docs/data-backfilling-to-storages#trigger-the-writing-to-the-storage) in the relevant flow.
  1. For long timeframes, consider breaking the process into smaller segments and repeating as necessary.

![Core Concepts - Data Backfilling - Storages](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Core%20Concepts%20-%20Data%20Backfilling%20-%20Storages.png)

## Trigger the Extraction

1. Go to the [**Sources**](https://app.dataddo.com/sources) page and click on your source.
2. Click on the **Manual Data Load** button next to your source.
  1. Select your date range. If the date picker is not present, temporarily [override the source configuration](/docs/data-backfilling-to-storages#override-source-configuration).
  2. Set the **Snapshot Keeping Policy** to **Replace**.
3. Click on **Load Data** to initiate the extraction.

![Historical data load - select date range](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Historical%20data%20load%20-%20select%20date%20range.png)

### Override Source Configuration

          
          

Once data backfilling is complete, please revert the source configuration to its original settings to ensure smooth regular data extractions.

When the calendar date picker is not available, you can override this by **adjusting the source URL**.

1. Go to the [**Sources**](https://app.dataddo.com/sources) page and click on your source.
2. Navigate to the **Advanced Settings** tab and locate the **URL** field.
3. **Change the date range** query parameters (for more details, see [the following section](/docs/data-backfilling-to-storages#change-date-range)).
4. **Save** the source configuration.

![Historical data load - source URL](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Historical%20data%20load%20-%20source%20URL.png)

### Change Date Range

The date range in your source URL can be indicated by e.g.

- **startDate** and **endDate**
- **since_send_time** and **before_send_time**
- **start** and **end**

In the source URL, look for strings such as **{{1d1}}** which represent a placeholders for the definition of the dynamic timeframe. It will look like the following example.

```
https://us16.api.mailchimp.com/3.0/campaigns?offset=0&count=500&since_send_time={{1d1}}&before_send_time={{1d1}}
```

**Replace the placehoders with**:

A static string representing a specific time frame, for example:

```
https://us16.api.mailchimp.com/3.0/campaigns?offset=0&count=500&since_send_time=2020-09-01T00:00:00+00:00&before_send_time=2020-09-14T23:59:59+00:00
```

A dynamic date range. In the example below we want to extract last 90 days' data, including today.

```
https://us16.api.mailchimp.com/3.0/campaigns?offset=0&count=500&since_send_time={{90d0}}&before_send_time={{90d0}}
```

For more information, see [**Setting Date Range**](/docs/setting-date-range).

## Trigger the Writing to the Storage

          **DATADDO TIP**

          

Before starting this operation, please **check the [write mode](/docs/data-storages#write-modes) of your flow**. We recommend using upsert, which either adds new records or updates existing ones, reducing the risk of duplicate entries.

1. Go to the [**Flows**](https://app.dataddo.com/flows) page and click on the **Manual Data Insert** button next to your flow.
2. **Confirm** the data insert and click on **Insert Data**.
3. Confirm the operation was successful by checking logs or inspecting the warehouse directly.

![Historical data load - load data to flow](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Historical%20data%20load%20-%20load%20data%20to%20flow.png)

---

## Troubleshooting

### Context Deadline Exceeded Error

```
rpc error: code = DeadlineExceeded desc = context deadline exceeded
```

This issue may be caused by extracting data over an extended timeframe. Please [override the source configuration](/docs/data-backfilling-to-storages#override-source-configuration) to a shorter timeframe.

### Cannot Change Write Mode to Upsert in Existing Flows

Changing to the [upsert](https://docs.dataddo.com/docs/data-storages#upsert) write mode for an existing flow is not possible for some specific destinations like [MySQL](/docs/mysql-destination). In these cases, Dataddo creates indexes to ensure smooth system operations and as such, write mode chages to existing flows are restricted.

**Use one of the following workarounds:**

1. If you can **lose the data** in the existing table without consequences or **recover it through backfilling**.
  1. Re-create the flow using upsert to ensure that the table and indexes are correctly established.
2. If **preserving** the data is crucial:
  1. **Delete** the existing flow.
  2. Add a **unique key index** on the columns you wish to include in the [**composite key**](/docs/data-storages#upsert). For example, using the SQL query below.
  3. Create a new flow with **upsert** as the write mode and a composite key that incorporates the column on which you created the unique index.

```
ALTER TABLE table_name
ADD UNIQUE INDEX unique_index_name (column1, column2);
```

Upsert inserts new rows and updates existing ones in the destination. By ensuring that the data in the table is updated if a matching record exists, or a new one is inserted, upsert prevents data duplication.