---
title: "Data Backfilling to Storages"
slug: "data-backfilling-to-storages"
description: "Load historical data from any date range to your storage with Dataddo. Easily load historical data from sources to destinations with one-time requests."
tags: ["Data warehouse", "Database", "Data flow", "Data backfilling"]
updated: 2025-03-23T16:48:03Z
published: 2025-03-23T16:48:03Z
canonical: "docs.dataddo.com/data-backfilling-to-storages"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataddo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Backfilling to Storages

Dataddo's **Data Backfilling** feature enables you to add historical data to [storage](/docs/data-storages){target="_blank"} solutions like [BigQuery](/docs/google-bigquery){target="_blank"}, [Snowflake](/docs/snowflake){target="_blank"}, or [Databricks](/docs/databricks){target="_blank"}. This makes your data storage comprehensive, including both current and past data, improving your analytics and reporting capabilities.

:::(Info) (**DATADDO TIP**)
See [Full Data Re-Sync](/docs/full-data-re-sync) to load **all your historical data at once**.

If your destination is a **dashboarding app** like [Looker Studio](/docs/looker-studio){target="_blank"}, [Power BI](/docs/power-bi){target="_blank"}, or [Tableau](/docs/tableau){target="_blank"}, refer to [**this article**](/docs/data-backfilling-to-dashboarding-apps){target="_blank"}.
:::

## The Process
1. Go to the [**Sources**](https://app.dataddo.com/sources){target="_blank"} page and [trigger the extraction](/docs/data-backfilling-to-storages#trigger-the-extraction) for your selected source and choose your date range.
    1. If the UI does not offer a date range picker, you can temporarily [override the source configuration](/docs/data-backfilling-to-storages#override-source-configuration).
2. Initiate the [data writing](/v1/docs/data-backfilling-to-storages#trigger-the-writing-to-the-storage) in the relevant flow.
    1. For long timeframes, consider breaking the process into smaller segments and repeating as necessary.

![Core Concepts - Data Backfilling - Storages](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Core%20Concepts%20-%20Data%20Backfilling%20-%20Storages.png){height="" width=""}

## Trigger the Extraction
1. Go to the [**Sources**](https://app.dataddo.com/sources){target="_blank"} page and click on your source.
2. Click on the **Manual Data Load** button next to your source.
    1. Select your date range. If the date picker is not present, temporarily [override the source configuration](/docs/data-backfilling-to-storages#override-source-configuration).
    2. Set the **Snapshot Keeping Policy** to **Replace**.
3. Click on **Load Data** to initiate the extraction.

![Historical data load - select date range](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Historical%20data%20load%20-%20select%20date%20range.png){height="" width=""}

### Override Source Configuration
:::(Warning)
Once data backfilling is complete, please revert the source configuration to its original settings to ensure smooth regular data extractions.
:::

When the calendar date picker is not available, you can override this by **adjusting the source URL**.

1. Go to the [**Sources**](https://app.dataddo.com/sources){target="_blank"} page and click on your source.
2. Navigate to the **Advanced Settings** tab and locate the **URL** field.
3. **Change the date range** query parameters (for more details, see [the following section](/docs/data-backfilling-to-storages#change-date-range)).
4. **Save** the source configuration.

![Historical data load - source URL](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Historical%20data%20load%20-%20source%20URL.png){height="" width=""}

### Change Date Range
The date range in your source URL can be indicated by e.g.
*  **startDate** and **endDate**
*  **since_send_time** and **before_send_time**
*  **start** and **end**

In the source URL, look for strings such as **{{1d1}}** which represent a placeholders for the definition of the dynamic timeframe. It will look like the following example.
```
https://us16.api.mailchimp.com/3.0/campaigns?offset=0&count=500&since_send_time={{1d1}}&before_send_time={{1d1}}
```
**Replace the placehoders with**:

A static string representing a specific time frame, for example:
```
https://us16.api.mailchimp.com/3.0/campaigns?offset=0&count=500&since_send_time=2020-09-01T00:00:00+00:00&before_send_time=2020-09-14T23:59:59+00:00
```
A dynamic date range. In the example below we want to extract last 90 days' data, including today.
```
https://us16.api.mailchimp.com/3.0/campaigns?offset=0&count=500&since_send_time={{90d0}}&before_send_time={{90d0}}
```
For more information, see [**Setting Date Range**](/docs/setting-date-range){target="_blank"}.

## Trigger the Writing to the Storage
:::(Info) (**DATADDO TIP**)
Before starting this operation, please **check the [write mode](/docs/data-storages#write-modes){target="_blank"} of your flow**. We recommend using {{glossary.upsert}}, which either adds new records or updates existing ones, reducing the risk of duplicate entries.
:::

1. Go to the [**Flows**](https://app.dataddo.com/flows) page and click on the **Manual Data Insert** button next to your flow.
2. **Confirm** the data insert and click on **Insert Data**.
3. Confirm the operation was successful by checking logs or inspecting the warehouse directly. 

![Historical data load - load data to flow](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Historical%20data%20load%20-%20load%20data%20to%20flow.png){height="" width=""}

***

## Troubleshooting
### Context Deadline Exceeded Error
```
rpc error: code = DeadlineExceeded desc = context deadline exceeded
```
This issue may be caused by extracting data over an extended timeframe. Please [override the source configuration](/docs/data-backfilling-to-storages#override-source-configuration) to a shorter timeframe.

### Cannot Change Write Mode to Upsert in Existing Flows
Changing to the [upsert](https://docs.dataddo.com/docs/data-storages#upsert){target="_blank"} write mode for an existing flow is not possible for some specific destinations like [MySQL](/docs/mysql-destination){target="_blank"}. In these cases, Dataddo creates indexes to ensure smooth system operations and as such, write mode chages to existing flows are restricted. 

**Use one of the following workarounds:**
1. If you can **lose the data** in the existing table without consequences or **recover it through backfilling**.
    1. Re-create the flow using {{glossary.upsert}} to ensure that the table and indexes are correctly established.
2. If **preserving** the data is crucial:
    1.  **Delete** the existing flow.
    2.  Add a **unique key index** on the columns you wish to include in the [**composite key**](/docs/data-storages#upsert){target="_blank"}. For example, using the SQL query below.
    3.  Create a new flow with **upsert** as the write mode and a composite key that incorporates the column on which you created the unique index.

```sql
ALTER TABLE table_name
ADD UNIQUE INDEX unique_index_name (column1, column2);
```