--- title: "CSV Universal Connector" slug: "csv-universal-connector" description: "Easily extract CSV data using one of Dataddo's Universal Connectors. Configure CSV files with custom delimiters, new line characters, offsets, and limits." tags: ["Data source", "Universal connector", "How-to guide"] updated: 2023-12-12T09:38:11Z published: 2023-12-12T09:38:11Z canonical: "docs.dataddo.com/csv-universal-connector" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.dataddo.com/llms.txt > Use this file to discover all available pages before exploring further. # CSV Universal Connector **CSV**, or **Comma-Separated Values**, is a plain-text file format used to store tabular data, where each line of the file represents a row of data, and the values in each row are separated by commas or other delimiters, like semicolons or tabs. CSV files are commonly used for data import/export, spreadsheet applications, and database management due to their simplicity and compatibility with various software systems. ## How to Create a CSV Data Source A comma-separated values (CSV) file is a delimited text file that **uses a comma to separate values** and each line of the file is a data record. This guide will be divided into the following steps: 1. [Fill in Basic File Information](/docs/csv-universal-connector#fill-in-basic-file-information) 2. [Optional Steps: Define Your CSV File](/docs/csv-universal-connector#optional-steps-define-your-csv-file) 3. [Specify Your HTTP Request](/docs/csv-universal-connector#specify-your-http-request) 4. [Preview Data and Save Your Data Source](/docs/csv-universal-connector#preview-data-and-save-your-data-source) If you need to authorize your connection, there are several possible ways to do so. Please refer to the [section on authorization methods](/docs/csv-universal-connector#authorization-methods). ### Fill in Basic File Information 1. On the **Sources** page, click on the [**Create Source**](https://app.dataddo.com/sources/new) button and select the connector from the list. 2. Select the [**HTTP method**](/docs/connectors#http-methods) to define a set of request methods. 3. Insert the **URL** of your CSV file. You can embed the service API authorization key or token directly in the call’s URL. For more information, refer to [the section on authorization](/docs/csv-universal-connector#authorization-methods). 4. If authorization is needed, **select your account**. 5. Select your [**snapshot keeping policy**](/docs/extraction#snapshot-keeping-policy). ### Optional Steps: Define Your CSV File 1. Select your **delimiter** if it is not the default comma. 2. Select how the **start of a new line** is defined. This can be defined in two ways: 1. Windows-based systems use `\r\n` 2. Unix-based systems (including Linux and macOS) use `\n` 3. If you need to divide the file into multiple consequent queries or when you need only a part of the file 1. Set the **file offset**. 2. Specify the **row limit**. (For more information, see the section on [how to use offset and limit](/docs/csv-universal-connector#how-to-use-offset-and-limit).) ### Specify Your HTTP Request 1. In the [**Transformation Pipeline**](/docs/transformation-pipeline) window, define a short script to process the data from these APIs. You can also click on the **Get transformation** button. In most cases, it will auto-generate a transformation script for you. ![Universal%20connector%20-%20get%20transformation](https://cdn.document360.io/084ed225-3f99-4644-a2da-39ca0cd5ef45/Images/Documentation/Universal%20connector%20-%20get%20transformation.png) 2. Define your [**HTTP headers**](/docs/connectors#http-headers), name-value pairs that will be included in the request and response messages of the HTTP protocol. In most cases, you will need to set this parameter **for authorization or setting the content type**. **DATADDO TIP** For more information, refer to [the section on authorization](/docs/csv-universal-connector#authorization-methods) or [our article on HTTP headers](/docs/connectors#http-headers). 3. **[Optional]** On the [**HTTP Body**](/docs/connectors#http-body) tab, you can see the data transmitted in an HTTP transaction message. 4. On the **Schedule** tab, choose your [**sync frequency**](https://docs.dataddo.com/docs/extraction#extraction-frequency) or the exact synchronization time. ### Preview Data and Save Your Data Source 1. Make sure your transformation script has **array symbols (square brackets [ ] )**. 2. Click on the **Get Data** button to test your transformation and see a preview table. 3. Click on **Save Source** and **congratulations, your new *data source* is ready!** ## Authorization Methods ### Embedding API Key in URL (Query String) Some services authorize API access via a **key embedded directly in the call’s URL**. Here's how you integrate the token into the API endpoint: ``` https://api.service.com?token=YOUR_TOKEN ``` ### Utilizing HTTP Headers for Static Tokens **HTTP headers** consist of name-value pairs positioned in the request and response messages of the HTTP protocol. They serve various functions, mainly authorization and defining the content type. For more information, refer to [the article on HTTP headers](https://docs.dataddo.com/docs/connectors#http-headers). #### Basic Authorization HTTP's built-in **basic authentication** requires sending an **Authorization** header with each request. This header includes the keyword `Basic`, followed by a **base64-encoded string** formatted as `username:password`. For example: ``` Authorization: Basic base64EncodedUsernameAndPassword ``` **Example** To authorize as `user123` / `p@ss123` the client would send the following. ``` Authorization: Basic dXNlcjEyMzpwQHNzMTIzLg== ``` When using universal connectors, input the appropriate credentials into your **HTTP Header** parameters to establish this authorization type. #### Bearer Authorization The **bearer token** system grants "bearer" access rights via **security tokens**. Requests must include an **Authorization header**, prefixed with `Bearer` and followed by the actual security token. For example: ``` Authorization: Bearer YOUR_TOKEN ``` Ensure you input the accurate token into your **HTTP Header** parameters when setting up Bearer authorization with universal connectors. --- ## Troubleshooting ### How to Use Offset and Limit Offset and limit parameters are useful when you need to divide the file into multiple queries or when you need only a part of the file. For example, due to a big data volume, you want to divide the data extraction into four batches. To do so, you will need to create four ***data sources***. 1. ***Data source*** 1: Offset = 0, Limit = 10 000 2. ***Data source*** 2: Offset = 10 000, Limit = 10 000 3. ***Data source*** 3: Offset = 20 000, Limit = 10 000 4. ***Data source*** 4: Offset = 30 000, Limit = 10 000 ## Related Articles Now that you have successfully created a ***data source***, see how you can connect your data to a dashboarding app or a data storage. **Sending Data to Dashboarding Apps** - [Simple Data Integration to Dashboards](https://docs.dataddo.com/docs/simple-data-integration-to-dashboards) - [Data Backfilling to Dashboarding Apps](https://docs.dataddo.com/docs/data-backfilling-to-dashboarding-apps) **Sending Data to Data Storages** - [Batch Ingestion to Data Warehouses](https://docs.dataddo.com/docs/ingestion-to-data-warehouses) - [Data Backfilling to Storages](https://docs.dataddo.com/docs/data-backfilling-to-storages) **Other Resources** - [Troubleshooting](https://docs.dataddo.com/docs/troubleshooting) - [Extraction Logs](https://docs.dataddo.com/docs/extraction-logs) - [Data Duplication](https://docs.dataddo.com/docs/data-duplication) A snapshot captures the state of data from a source at a specific point in time. Each data extraction creates a new snapshot and all snapshots collectively form a time series. A source is a collection of data from an authorized service that's been connected via a Dataddo connector. Data within the source is automatically refreshed based on the source's configuration.