--- title: "Release Note 2024-11" slug: "release-note-2024-11" description: "Dataddo is a secure, scalable ETL platform that simplifies data integration and automation. See our Nov Release Note for the latest updates and enhancements." updated: 2024-11-23T20:04:45Z published: 2024-11-23T20:04:45Z --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.dataddo.com/llms.txt > Use this file to discover all available pages before exploring further. # Release Note 2024-11 We’re excited to bring you the latest updates to Dataddo! The **November 2024** release continues our commitment to optimizing the **Flows** page, introducing new improvements designed to provide a clearer, more comprehensive understanding of your data at a glance. Alongside these enhancements, we’ve also made key adjustments to streamline your experience and addressed various bugs for improved performance. Here’s a detailed overview of the new features, UI updates, and bug fixes included in this release. ## Connectors **New Connectors:** - DynamoDB - LocalFalcon - Survicate - Zenvoices **New Datasets:** - **Instagram Media**: Added the **Comments** dataset. - **LinkedIn Organic**: Added the **Audience Insights** dataset. **Connector Updates:** - **TikTok**: The Dataddo Hash field is now available. - **YouTube Analytics**: The **Videos - Retention** dataset received a new emitter and updates, with corresponding updates applied to all sources using this dataset in Dataddo. ## New Features ### Full Data Re-Sync Load historical data all at once with our **new full data re-sync functionality**. Previously, users needed to determine the maximum data load for each connector and manually chunk data, which could be challenging. The system now automatically determines the limitations and optimally distributes the requests to sync all available data efficiently. **Conditions for Use:** - Users must have `write` permissions (e.g., the user can and has configured upsert write mode for some flows). The system will verify this before enabling the feature. This functionality is being implemented gradually as each connector has different max loads and is designed to eventually replace the ad-hoc manual data load process. ### Sensitive Data Handling The **advanced settings** have been streamlined to enhance data **security and usability**. A new source set-up step will soon enable users to define **how detected sensitive data is handled**, including options for **hashing sensitive data** for added security. Previously, advanced settings allowed **only adding metadata** such as Property ID, Dataddo Insert Date, and Dataddo Hash (a unique identifier or composite key for upsert). ![2024-11 Sensitive data](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/2024-11%20Sensitive%20data.png) ## UI Changes and Enhancements ### Ongoing Enhancements to the Flows Page We continue to enhance **flow-level data movement capabilities**, delivering improved observability and clearer, at-a-glance insights into the origins, destinations, and pathways of your data. **Flow Grid Write Modes:** - The flow grid now displays write modes more comprehensively, including **streaming mode**, in addition to upsert, truncate, and truncate-insert. This enhancement provides greater transparency and helps maintain data quality. ![2024-11 Streaming](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/2024-11%20Streaming.png) **Destination Descriptors:** - The flows page now includes destination descriptors, allowing you to see which flow is sending data to specific data warehouses (e.g. identifying different projects, databases, and tables within GBQ) all on one page. ![2024-11 Destination descriptor](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/2024-11%20Destination%20descriptor.png) ### Comprehensive Schema Overview **Fixed-schema connectors** are also being continuously improved to provide a more comprehensive view of schema details. This will allow users to better understand the data they’re working with. Apart from the full list of available attributes, the source setup now displays: - **Data types** for each field. - **Tooltips** explaining each field and the type of data being extracted. - **Indicators for sensitive data**, tying into the upcoming sensitive data handling feature. - **Attribute relations and dependencies** (viewable by hovering over the attribute name), showing connections to other datasets to help: - Identify any additional required data for joins. - Address data gaps for a clearer understanding of how data works together. ![2024-11 Comprehensive schema overview](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/2024-11%20Comprehensive%20schema%20overview.png) **In development:** - The **enforcement of correct data types** will be gradually implemented across individual connectors to ensure system stability. - The comprehensive dataset view is being **progressively rolled out to all fixed-schema connectors**, expanding beyond the 50% of templates currently covered. These updates aim to give users a clearer and more actionable understanding of their data structure and relationships. ### Additional UI Updates **New Dashboard Controller:** - A new feature displays the number of **broken** authorizers, sources, destinations, and flows. Users can click on **Debug** to view a detailed list of the affected components and repair them. ![2024-11 New Dashboard](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/2024-11%20New%20Dashboard.png) **Snapshotting Tab:** - Date range selection is now conveniently displayed on the source details page, making it easier to review and manage snapshotting configurations. ![2024-11 Date range expression](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/2024-11%20Date%20range%20expression.png) ## Platform Optimization To ensure smoother performance and optimize system efficiency, **only sources actively used in flows will now be regularly synced**. This update allows us to focus resources on syncing the data that matters most, improving reliability and overall platform performance. ### In Development We are actively preparing additional enhancements to streamline data movement observability and management: **Enhanced Flow Type Information:** - Flow types will soon feature distinctive icons, and eventually, the write mode (append or replace) will be displayed directly at the flow level. This information will be moved from the source level to the flow level for greater clarity. - All data movement configuration details will ultimately be consolidated on the Flows page. **Enhanced Source Configuration Insights:** - The Sources page will include what account the data is extracted from (e.g., identifying specific HubSpot accounts in multi-account setups) and the associated configuration settings. The rollout of descriptors will be gradual, with custom implementations tailored to each connector. Stay tuned as we continue refining these functionalities to further optimize your data management experience. ## Bug Fixes **Podbean, Google Analytics 4, ActOn, and LinkedIn:** - Resolved reauthorization issues where users received an error notification indicating the account could not be found. The issue was identified and fixed within 24 hours of detection. **TikTok:** - Fixed an issue where connectors showed successful extractions, but no data was retrieved. The connector was promptly updated to resolve the problem. **LinkedIn Organic:** - Added missing facets to the Audience Insight dataset for improved targeting insights: - `urn:li:adTargetingFacet:companyCategory` - `urn:li:adTargetingFacet:jobFunctions` **Google Merchant Center:** - Resolved a pagination issue by implementing a cursor-based pagination parameter (`pageToken`). The connector now successfully paginates results with each API call using the updated loader file.