Data flows

Data flows allow you to orchestrate the integration of your data from various online services to your desired destinations including BI tools such as Tableau or Google Data Studio, as well as to big data storages such as Google Big Query or AWS Redshift, file storages or SQL-based data warehouses. Within a single data flow, you can include data from multiple data connections. This is particularly handy for organizations who handle multiple social network accounts those that need to merge data from different services together for a consolidated output.

Data connection orchestration

Each data flow may contain one or multiple data connections. However, you must make sure that all the sources have the same structure, i.e. they have the same number of columns containing the same data types.

Example 1: Allowed orchestration

The following data connections can be orchestrated within a single data flow. Both sources have the same number of columns and the same order of data types.

Source 1

Column name

Column data type

column A

DATE

column B

STRING

column C

STRING

column D

FLOAT

 

Source 2

Column name

Column data type

column E

DATE

column F

STRING

column G

STRING

column H

FLOAT

 

Example 2: Forbidden orchestration

The following data connections cannot be orchestrated within a single data flow since Source 2 has a different count of columns than source 1.

Source 1

Column name

Column data type

column A

DATE

column B

STRING

column C

STRING

column D

FLOAT

 

Source 2

Column name

Column data type

column E

DATE

column F

STRING

column G

STRING

 

Example 3: Forbidden orchestration

The following data connections cannot be orchestrated within a single data flow. Although both sources have the same number of columns, they differ in column types.

Source 1

Column name

Column data type

column A

DATE

column B

STRING

column C

STRING

column D

FLOAT

 

Source 2

Column name

Column data type

column E

DATE

column F

STRING

column G

STRING

column H

STRING