Data Transformations

Dataddo has built its platform to provide such transformative capabilities, ensuring data is not just available but is also accurate, useful, and analytics-ready. Here's an in-depth look at the data transformation features at every stage of the data processing journey.

A Multi-Layered Approach to Data Transformations

Dataddo's multi-layered approach to data transformations, encompassing extraction-level, flow-level, and destination-level stages, offers a strategic balance between automation and flexibility.

This structured method reduces the need for extensive transformations at the data warehouse or final destination stages. By tackling key transformation tasks early in the process, it ensures that data is consistently analytics-ready, regardless of its origin. This not only simplifies the data integration process but also accelerates the journey from raw data to actionable insights.

Extraction-Level Transformations

By default, Dataddo's connectors automatically perform extraction-level transformations to make data analytics-ready in a fully no-code manner. This includes operations such as

• Data flattening,
• Harmonization,
• PII exclusion, or
• Metadata addition.

For those with specialized needs, Dataddo offers universal connectors (e.g. JSON Universal Connector or CSV Universal Connector), which allow users to seamlessly blend automated procedures with tailored data extraction requirements.

Data Flattening

Raw data, especially from APIs, frequently arrives in nested or hierarchical formats. Dataddo automatically simplifies this intricate structure, transforming it into a more accessible tabular format, all without requiring any coding on the user's part. This ensures a seamless no-code experience for users, allowing for quick and efficient data integration.

Data Harmonization

Ensures that data is analytics-ready and machine-readable, regardless of its original format. This process standardizes various data elements, converting dates to consistent formats, and ensuring that numerical values, like integers or floats, align with their correct data types. By streamlining these discrepancies, Dataddo guarantees that data feeds into analytics tools smoothly and without the need for additional transformation.

Metadata Inclusion

Beyond the main data, meta-information like the Dataddo Extraction Timestamp is crucial for tracking Slowly Changing Dimensions (SCD) and understanding the lineage of data.

PII Exclusion and Hashing

In the realm of extraction-level transformations, Dataddo prioritizes data security and compliance. To safeguard Personally Identifiable Information (PII), Dataddo offers PII Exclusion, ensuring sensitive data is omitted during extraction. If identifiable details are essential, Hashing is utilized. This process converts PII into a unique string of characters, allowing analysis without compromising privacy.

Flow-Level Transformations

Data Union

At the flow-level transformations, Data Union serves a specific role. It is akin to the UNION operator in SQL language, combining datasets from different sources. However, for Data Union to function correctly, it's imperative that the sources have an identical schema in terms of columns and their respective data types. This ensures seamless integration and consistency across the unified dataset. Moreover, employing Data Union in Dataddo is a fully no-code experience, making the process intuitive and accessible for all users.

Data Blending

Another powerful tool at the flow-level transformations within Dataddo. Analogous to the JOIN operation in SQL, it lets users integrate data from multiple sources based on common identifiers or keys. This allows for the creation of enriched datasets by pulling complementary data from different sources into a single view. Just like other Dataddo functionalities, Data Blending is presented as a fully no-code experience, ensuring that users, regardless of their technical proficiency, can effortlessly merge data from various sources.

Destination-Level Transformations

At this stage, transformations depend on the destination platform's capabilities. Data warehouses typically offer a broad array of transformation tools, from aggregations to pivots, due to their robust data processing capabilities. Conversely, dashboarding applications may emphasize visual transformations for analytical presentation. While Dataddo delivers data in an analytics-ready format, users should familiarize themselves with their destination's inherent transformation tools to maximize its utility, whether it's a data warehouse like Snowflake or a visualization platform like PowerBI.