Dashboarding Apps: A Technical Overview

Directly connecting to dashboarding apps like Looker Studio, PowerBI, or Tableau is a highly effective approach for simpler use-cases within Dataddo, especially when dealing with smaller datasets. This strategy provides a streamlined pathway from your data sources to your visualization platforms, thus eliminating the need for an intermediary data warehouse. This makes dashboarding apps an ideal solution for uncomplicated data visualization tasks involving less volume and complexity. Nevertheless, for more complex data processes or larger volumes, a data warehouse might be a more appropriate destination.

Architecture considerations

Direct data delivery to dashboarding apps is a feature supported by Dataddo, eliminating the need for an intermediate data warehouse or database. This is facilitated by our embedded SmartCache storage system, a straightforward solution ideal for scenarios not requiring extensive data volumes or complex data transformations.

Nevertheless, SmartCache is not a replacement for a data warehouse. If your data source exceeds 100,000 rows or if your data operations extend beyond simple joins, such as Data Blending and Data Union, we recommend opting for a data warehouse solution.

Smart Cache

Data Residency in SmartCache

With Dataddo's SmartCache, your data is conveniently stored on our servers, eliminating the need to provision a data warehouse or database. However, you maintain control over the physical location of your data. Dataddo currently supports up to 16 global locations for data storage. You can specify your preferred data residency by navigating to Account Details and setting Data Residency, ensuring your data remains within a desired jurisdiction or region.

Ensuring Data Security with SmartCache

For a comprehensive overview of our security measures, please refer to our Technical & Organizational Security Measures.

Data Encryption at Rest

Dataddo ensures the security of your data at rest through automated encryption using Advanced Encryption Standard (AES) 256. This strong encryption method secures your data on our storage volumes. Additionally, for an extra layer of control and security, you have the option to bring and use your own encryption keys. The keys themselves are safeguarded by a third-party Hardware Security Module (HSM)-backed key management service to ensure maximum protection.

Data Encryption in Transit

Dataddo prioritizes the safety of your data even while it's in transit. All network traffic, which includes data sent to dashboards and other apps, is securely encrypted using Transport Layer Security (TLS). This robust protocol ensures that your data remains secure and intact during transmission, further fortifying our commitment to data security.

Data Volume Constraints

Dashboarding apps can potentially experience slower performance when processing extremely large volumes of data, and in some cases, may require additional data processing. Generally, if your dataset exceeds 100K rows per source, it would be advantageous to consider integrating an external storage solution into your data architecture for optimal performance.

Storage Modes

To optimize the data management process, Dataddo offers two distinct storage modes Append and Replace. These modes define how new data is managed with each synchronization, giving users control over the maintenance of their datasets.

• Append. In this mode, the Smart Cache retains the data with each synchronization. Instead of overwriting existing data, new data fetched during the synchronization process is added to the existing dataset. This is beneficial when historical data tracking is essential, as it accumulates and maintains a comprehensive history of your data over time.

• Replace. Contrary to the Append mode, the Replace mode overwrites the existing data in the Smart Cache with fresh data fetched during each synchronization. This mode ensures that your storage contains the most recent snapshot of your data, making it ideal for scenarios where maintaining the most current data is more important than historical tracking.

Estimating Required Data Flows

If you need to add more than one data source to your dashboard, there are two possibilities.

1. One flow: The data from different sources that can be joined in one table either via data blending (through a join key, for max 2 sources) or data union (if table schemas are exactly the same, for 2+ sources).
2. Two or more flows: If your sources don't have a join key and different table schemas, you will need to create one flow per source.

Troubleshooting

Data duplicates in the dashboard

Causes & solutions

1. Inapproprite Source snapshotting. If you're encountering duplicates, one cause could be the snapshotting policy of your sources. To address this, verify the attached sources and their respective Snapshot Keeping policies. If preserving historical data is not necessary, consider switching to the Replace snapshotting policy.
2. Data Overlaps Across Extractions. If your Snapshot Keeping policy is set to Append and the data overlaps with each extraction (e.g., you're extracting data for the entire week or month each day), consider adjusting the configuration of your Dynamic Date Range or setting the snapshotting policy to Replace.