- 2 Minutes to read
- DarkLight
Setting Date Range
- 2 Minutes to read
- DarkLight
When configuring a time-dependent data source, you can set the date range. Each time a snapshot is taken, Dataddo will load data based on the specified date range.
Use Case Examples
Daily Data Loads
Set the date range to Yesterday and the snapshotting frequency to Daily.Monthly Data Loads
Use Last Month as the date range and set the snapshot frequency to Monthly.Specific Use Case
To extract weekly Facebook Ads data and send it to MySQL, set the date range to Last 7 days and the snapshot frequency to Weekly when creating a source. This will capture a snapshot of the past week's data every week.
You can load data from the last couple of days, last month, last year etc. This is especially useful when e.g. loading historical data
Dynamic Date Range
If the date range you need is not available in the date picker selection, you can specify a different date range in the source URL (in the Sources page, click on your source and navigate to the Advanced Settings).
In these cases, your date-related parameters can be defined as part of the API call using a dynamic date range, which extracts data relative to today’s date.
The URL of a source can look like this:
https://api.service.com/endpoint?startDate={{1m1}}&endDate={{1m1}}
The expression enclosed in double curly brackets {{1m1}} consists of three parts:
- The number at the start of expression 1m1. This represents the number of units (here months) to go back from the current date and defines the start of the range. 0 represents the current value.
- Unit definition: d = day (1d0), m = month (1m1), w = week (1w1)
- The last number sets the end of the range. 0 marks today, 1 marks yesterday, and so on. There is a rule that the first digit has to always be larger than the second one.
Expression | Meaning in ISO 8601 |
{{0d0}} | Today |
{{1d1}} | Yesterday |
{{1d0}} | Yesterday and today |
{{0m0}} | This month |
{{1m1}} | Last month |
{{90d1}} | 90 ago until yesterday |
{{1w1}} | Last week |
{{0w0}} | This week starting from Monday |
{{0y0}} | This year |
{{1y1}} | Last year |
{{1d1|U}} | Yesterday in Unix seconds |
Examples
- API Call with 'yesterday' Date Range
https://api.service.com/endpoint?startDate={{1d1}}&endDate={{1d1}}
- API Call with 'this month' Date Range
https://api.service.com/endpoint?startDate={{0m0}}&endDate={{0m0}}
Date String Formatting
Date string is by default written in the YYYY-MM-DD format (e.g. 2020-12-31). Unfortunately, not all APIs follow ISO 8601 standards. In such cases, it is possible to use date string formating.
Custom Formatting Examples
Expression | Output |
{{0d0|Ymd}} | 20201231 |
{{0d0|Y-m-d\TH:i:s}} | 2020-12-31T00:00:00 |
Change Date Range in Manual Data Load
Dynamic date range is also used for manual data load when a calendar date range is not available. In this case, to manually load historical data, you will need to specify your date range in the source URL.
Static Date Range
Using a static date range expression is recommended mainly for historical data loads when you need to extract data from a specific time period.
https://api.service.com/endpoint?startDate=2020-01-01&endDate=2020-12-31
Please keep in mind that if this URL is not changed after your historical data load back to a dynamic date range, you will be extracting data will be from January 1, 2020 until December 31, 2020 every time.
For more information on manual data load and data backfilling, please see the following articles: