Data Flows
  • 1 Minute to read
  • Dark
    Light

Data Flows

  • Dark
    Light

Data Flows allow you to orchestrate the integration of your data from various online services to your desired destinations including BI tools such as Tableau or Google Data Studio, as well as too big data storage such as Google Big Query or AWS Redshift, file storages or SQL-based data warehouses. Within a single data flow, you can include data from multiple data connections. This is particularly handy for organizations that handle multiple social network accounts those that need to merge data from different services together for a consolidated output.

Data Connection Orchestration

Each data flow may contain one or multiple data connections. However, you must make sure that all the sources have the same structure, i.e. they have the same number of columns containing the same data types.

Example 1: Allowed Orchestration

The following data connections can be orchestrated within a single data flow. Both sources have the same number of columns and the same order of data types.

Source 1

Column name Column data type
column A DATE
column B STRING
column C STRING
column D FLOAT

Source 2

Column name Column data type
column E DATE
column F STRING
column G STRING
column H FLOAT

Example 2: Forbidden Orchestration

The following data connections cannot be orchestrated within a single data flow since Source 2 has a different count of columns than source 1.

Column name Column data type
column A DATE
column B STRING
column C STRING
column D FLOAT

Source 2

Column name Column data type
column E DATE
column F STRING
column G STRING

Example 3: Forbidden Orchestration

The following data connections cannot be orchestrated within a single data flow. Although both sources have the same number of columns, they differ in column types.

Source 1

Column name Column data type
column A DATE
column B STRING
column C STRING
column D FLOAT

Source 2

Column name Column data type
column E DATE
column F STRING
column G STRING
column H STRING

Was this article helpful?

What's Next