Manage and Monitor Data Flows

To ensure your data quality is maintained, Dataddo offers multiple ways to monitor your data. These methods can be divided into two categories:

• In-app monitoring: In-app notifications and statuses.
• Monitoring via Dataddo Logs connector: Extract information about Dataddo logs and send it to your data destination.

In-App Monitoring

Flow Monitoring

Understanding the status of your data flow in Dataddo is crucial for managing your data effectively. This section clarifies the four distinct flow statuses you'll encounter:

1. Live (green): Your flow becomes live once it's created and will stay live with every successful data extraction attempt.
2. Reconnecting (orange): If a data extraction attempt fails, your data flow will try to reconnect three times in total. If the subsequent attempt is successful, it will return to Live state, otherwise it will become Broken.
3. Broken (red): Your flow will be marked as broken if the extraction fails for the third time.. Please restart your flow to fix the issue or get more details.

Restart a Flow

Restarting a broken data flow can help resolve issues or identify the underlying problem.

1. On the Flows tab, click on the Restart Flow icon next to your broken flow. Dataddo will check if the connection to your source and destination is working correctly or if you need to reauthorize your service.
2. If the connection test is successful, click on Load Data to load missing data.
3. Type CONFIRM and click on Insert data.

Monitoring via Dataddo Logs Connector

The Dataddo Logs connector allows you to extract comprehensive log data seen within the app, to efficiently monitor your data management process.

Create a data source using the Dataddo connector and send the following data to any data destination:

• Authorizer logs
• Data extraction logs
• Data load logs

Authorizers

The Authorizers dataset provides insights, such as:

• Exact time of authorizers' latest use
• Status of the authorizer

Refer to our website for the full list of metrics and attributes.

Data Extraction

To monitor your extraction processes, the Dataddo connector offers the following datasets.

The Sources and extractions dataset provides information such as:

• Exact time of source's latest data extraction
• Number of extracted rows per source

The Extraction logs dataset allows you to get information such as:

• A list of all extraction attempts
• Extraction attempt outcome
• Number of retrieved rows per extraction

Refer to our website for the full list of metrics and attributes.

Data Loads

To monitor your data writing operations, the Dataddo connector offers the following datasets.

The Flows to dashboards and BI apps dataset provides information such as

• Exact time of the latest data load to your dashboard
• Flow status (live, broken, reconnecting)

The Flows to storages and applications dataset will offer information such as:

• Exact time of the latest data load to your warehouse
• Status of the flow

The Write logs dataset will provide insights such as:

• A list of all data load operations
• Data load outcome
• Number of rows written per data load

Refer to our website for the full list of metrics and attributes.