Dynamic File Naming Patterns

Dataddo supports dynamic file naming when creating data sources and data flows, giving you control over how files are created using variables like dates or object identifiers.

Depending on the use case, this feature serves two purposes:

• File Naming Patterns: Used when configuring data sources to locate and extract the correct input file.
• File Partitioning: Used in data flows to organize output data into multiple files based on time or other variables.

File Naming Patterns (for Data Sources)

When setting up a file-based data source (e.g., from Microsoft OneDrive as a Source), you can define a dynamic file name to automatically match files that update daily or periodically.

This is ideal for systems that generate a new file each day and can embed the date in the file name.

Dynamic Variables for Sources

Below, you will find all supported dynamic variables for data sources.

You can change the date format using date string formatting, e.g.:

• {{today|Y-m-d}} → 2020-01-31
• {{yesterday|d_m_Y}} → 31_01_2020

Example
For a file name like sales_data_20200101.csv, you can use the following name pattern:

sales_data_{{today|Ymd}}.csv

File Partitioning (for Data Flows)

File partitioning splits large datasets into smaller, manageable partitions, based on criteria like date. This technique enhances data organization, query performance, and management by grouping subsets of data with shared attributes.

When configuring a data flow that outputs data to a file-based destination (e.g., SFTP Server or Microsoft OneDrive), you can define a custom file name pattern to partition the output.

Example
You can use either dynamic variables or dynamic date ranges in your file names and change the the date format using date string formatting, e.g.:

• {{today|Y-m-d}} → 2020-01-31
• {{yesterday|d_m_Y}} → 31_01_2020
• xyz_{{1d1|Ymd}} → xyz_20200131

Dynamic Variables for Flows

Below, you will find all supported dynamic variables for data flows.

Dynamic Date Range for Flows

Use expressions like {{1m1}} (last month) or {{90d1}} (90 days ago through yesterday) to define custom ranges. The syntax {{<start><unit><end>}} breaks down as:

1. Start: Number of units back from today (e.g., 1 month ago).
2. Unit: d=day, w=week, m=month, y=year.
3. End: Number of units back marking the range’s end (must be ≤ start).

Date String Formatting

Specify the date string format used in your dynamic file name by applying standard strftime patterns. This determines how the date appears when inserted into the file name.