Amazon S3 (Simple Storage Service) is an object storage service by Amazon Web Services. The AWS S3 connector reads a file from a bucket and turns it into a Dataddo data source, so you can send its content to any dashboard, database, or data warehouse.
Authorize Connection to AWS S3
Before you can create a data source, connect Dataddo to your S3 bucket.
- On the Authorizers page, click on Authorize New Service and select AWS S3.
- Fill in the following fields:
| Field | Details |
|---|---|
| Label | A name for this authorizer in Dataddo. |
| Bucket | The identifier of the S3 bucket you want to read from. |
| Region | Region of the S3 bucket. |
| Key | The AWS access key ID. |
| Secret | The AWS secret access key. |
- Click on Save. Dataddo validates the connection before the authorizer is created.
Supported File Formats
The connector reads one file per data source, in any of these formats:
| Format | Notes |
|---|---|
| CSV | Set the CSV Delimiter, whether the file has a Header row, an optional Comment character (lines starting with it are skipped), and Lazy Quotes for files with imperfect quoting. |
| JSON | An array of objects, one object per row. |
| XML | Repeated elements become rows. |
| Parquet | Columnar files exported from data platforms. |
Files can be plain or GZIP-compressed; set File compression accordingly.
Selecting the File
When you create the source, point it at the file:
- Path - the directory that holds the file.
- File name - the name of the file to read.
The path and file name support dynamic date placeholders, so one source can follow files that are named by date. {{today}} and {{yesterday}} are replaced at run time; add a date format after | (the default is Ymd):
| File name mask | Resolves to |
|---|---|
report_{{today}}.csv |
report_ followed by the current date, e.g. report_20260709.csv |
export_{{yesterday\|Y-m-d}}.csv |
export_ followed by yesterday's date, e.g. export_2026-07-08.csv |
How Data Extraction Works
There is no date range for this connector. Every run downloads the file at the configured location (after resolving the date placeholders) and extracts its full current content.
The two most common setups:
- One file that is updated in place: schedule the source as often as the file changes and use the replace write mode in the flow, so the destination table mirrors the file.
- A new date-stamped file every day: use a
{{yesterday}}mask in the file name, schedule the source daily, and use the append write mode. Each run adds one day's file, which builds the history over time.
Transformation
Before the data reaches your destination, an initial transformation turns the parsed file into rows. Dataddo pre-fills it based on the selected file format and header settings. You can edit it in the Transformation editor when creating the source, for example to rename fields, unwind nested arrays, or drop columns.
How to Create an AWS S3 Data Source
- In Dataddo, open Sources and click Create Source.
- Select AWS S3 and choose the Authorizer you created above.
- Fill in the path and file name, using date placeholders if the file is named by date.
- Select the File Format and its options (delimiter and header for CSV), and the File compression if the file is gzipped.
- (Optional) Adjust the pre-filled Transformation.
- Click Test Data to preview the result, then click Save.
Troubleshooting
Data Preview Unavailable
No data preview when you click on Test Data might be caused by an issue with your source configuration. The most common causes are:
- Date range: Try a smaller date range. You can load the rest of your data afterward via manual data load.
- Insufficient permissions: Please make sure your authorized account has at least admin-level permissions.
Related Articles
Now that you have successfully created a data source, see how you can connect your data to a dashboarding app or a data storage.
Sending Data to Dashboarding Apps
Sending Data to Data Storages
Other Resources