Schema

Each data source has its own data schema, which signifies a structured arrangement of columns. This schema provides the blueprint for organizing and managing data within each data source.

Columns in a schema

• Are defined by a unique data type and label
• Cannot be altered in order to safeguard consistency and prevent undesired effects within downstream systems, such as data warehouses

Supported Data Types

To ensure maximum data type compatibility across a wide range of systems, Dataddo defines data types in a basic, yet highly compatible manner. The platform currently supports four primary data types.

• Integer: Stores whole numbers, which can be positive, negative, or zero. Dataddo supports up to 64-bit integers to accommodate a wide range of numerical values and ensure maximum compatibility.
• Float: Stores double-precision floating-point numbers with up to 17 significant digit which can be both positive and negative. Dataddo supports up to 64-bit floats for maximum compatibility.
• String: Represents text and stores complex types like objects. String can comprise a set of characters including spaces and numbers. Dataddo allows string values with up to 16MB capacity to ensure broad compatibility.
• Date: Supports datetime information with up to nanosecond precision for maximum compatibility.

Changing Data Types

Modifying the data types of your source columns is a delicate operation, which may disrupt your data flows or induce data loss. When a data type change is initiated, Dataddo creates a new table in the corresponding database, which could lead to inconsistencies if not handled carefully.

1. Go to the Sources page in Dataddo and click on your data source.
2. Navigate to the Schema tab.
3. Here, you can change your column names and data types.

Changing Column Labels

Edit the field names by clicking on your data source. Navigate to the Schema tab and rename your field. Please make sure you change the names in your database as well.

Troubleshooting

Unable to Add a New metric, dimension, or Attribute to an Existing Source

You cannot add a new metric, dimension, or attribute directly through the data source edit, as this action would require altering the schema. A simple solution would be cloning the data source and configuring it with different metrics.

Data Type Mismatch

Always make sure to update the data types in both Dataddo and your destination database to maintain consistency. Failing to do this could result in errors or incomplete data transfers. If your database doesn't allow editing data types, you'll need to create a new data source or data flow.

Loss of Precision

Converting from a float to an integer type will round off the decimal places, changing your data values. For example, 5.4 will become 5. As such it is typically recommended to change to float rather than vice versa.

Columns with Unsupported Data Type

If your original table contains columns with an unsupported data type, e.g. binary you will be unable to select these columns for data extraction.
If your data destination is a data storage, you can use CAST to change your data type to a supported one through a custom statement.

SELECT
    CAST(columnname AS int) AS new_columnname,
    other_columns
FROM
    tablename;