- 1 Minute to read
-
DarkLight
Data Duplicity
- 1 Minute to read
-
DarkLight
Check Records in Your Source
If one record has multiple values from fields, that are usually for single value, such as two email adresses, phone numbers or orders, the record will be recorded twice in the table but the second one will contain the second values. This will appear as a duplicity.
Check Destination's Writemode
If the write mode for your destination is INSERT, it may cause duplicities in the table. For example, you can select UPSERT write mode instead.
Incorrect Unique Columns for UPSERT
When creating a flow and configuring the destination, for UPSERT write mode, make sure to select the correct Unique Columns. Otherwise, instead of updating the existing records and inserting the new ones, the records will be inserted again, as the unique column was not found.
If you chose Customer ID as a unique column, the system will first check if the record already exists and updates in case of a change. However, if you choose Login time as the unique column, this value will not be the same and the record will be then duplicated. Read more about write modes.
Check Your Snapshotting Policy
Some combinations of snapshotting settings can lead to duplicities. Make sure you chose the right combination od Date range and Sync frequency.
If you select Last 7 days for the Date Range, and Daily for Sync Frequency, you will have data of last seven days inserted each day, meaning you will have 7 times more data than you want.
Feel free to contact us and we will help you with the setup. To speed the process of resolving your ticket, make sure you provide us with sufficient information.