Dashboarding Apps: A Technical Overview

Directly integrating with dashboarding applications like Looker Studio, PowerBI, or Tableau provides a streamlined solution for simple data visualization tasks within Dataddo.

This method is particularly suitable for smaller datasets as it facilitates direct data transfer from your data source to visualization platforms, eliminating the need for an intermediary data warehouse. However, for larger data volumes or more intricate processes, considering a data warehouse is advisable.

Architecture Considerations

Direct data delivery to dashboarding apps removes the requirement for an intermediary data warehouse or database. This is enabled by our embedded SmartCache storage system, making it an ideal choice for tasks not demanding high data volume or complex data transformations.

However, it's essential to note that SmartCache is not a substitute for a data warehouse. If your data volume surpasses 100,000 rows or if you require advanced operations beyond simple joins like data blending and data union, a data warehouse solution is recommended.

Smart Cache

Data Residency
Ensuring Data Security
Handling Large Data Volumes
Storage Modes

Data Residency in SmartCache

With Dataddo's SmartCache, your data is stored on our servers, eliminating the need for an external data warehouse or database. You retain control over your data's physical location. Dataddo currently supports 16 global storage locations. Set your preferred data residency in Account Details to guarantee data stays within a specific jurisdiction or region.

Ensuring Data Security with SmartCache

For a comprehensive overview of our security measures, refer to our Technical & Organizational Security Measures.

Data Encryption at Rest

Dataddo ensures the security of your data at rest through automated encryption using Advanced Encryption Standard (AES) 256. This strong encryption method secures your data on our storage volumes.

For enhanced control and security, you can opt to provide and use your own encryption keys which safeguarded by a third-party Hardware Security Module (HSM)-backed key management service.

Data Encryption in Transit

Dataddo ensures data security during transit. All network traffic, including data directed to dashboards and other apps, is encrypted using Transport Layer Security (TLS), emphasizing our commitment to data safety. This robust protocol ensures that your data remains secure and intact during transmission.

Handling Large Data Volumes

Dashboarding apps might experience slower performance when processing large data volumes. If your dataset exceeds 100,000 rows per source, integrating an external storage solution ensures better performance.

Storage Modes

To optimize the data management process, Dataddo offers two storage modes (also called snapshot keeping policy or simply snapshotting policy): Append and Replace. These modes determine how new data is handled during synchronization, offering users flexibility in data management.

• Append: This mode retains data in the SmartCache with each sync, adding new data to existing sets. It's ideal for tracking historical data as it accumulates and maintains a comprehensive history of your data over time.

• Replace: This mode overwrites existing data in the SmartCache with fresh data during each sync. Replace storage mode is suitable when having the most recent data snapshot is a priority.

Estimating Required Data Flows

If you need to add more than one data source to your dashboard, there are two possibilities:

1. One flow: Data from different sources can be joined in one table via
   1. Data blending: Data blending is done through a join key and is possible for max 2 data sources.
   2. Data union: Data union is possible for more than 2 data sources, however, all table schemas have to be the same.
2. Multiple flows: If data sources lack a join key or have different table schemas, create a separate flow for each source.

Troubleshooting

Addressing Data Duplicates

Data duplication in dashboards can arise due to:

1. Inappropriate source snapshotting: Review the snapshotting policy of your data sources. If historical data isn't a priority, switch to the replace storage mode.
2. Data overlaps: If you your snapshot keeping policy is set to append and you notice data overlaps with each extraction, you can
   1. Modify your dynamic date range configuration.
   2. Switch the snapshot keeping policy to replace.