SFTP, or Secure File Transfer Protocol, is a network protocol used for securely transferring files between a client and a server. An SFTP server is a server that supports this protocol, enabling secure and encrypted file transfers over a network, making it a safer and more protected alternative to traditional FTP for data exchange.
Authorize Connection to SFTP server
Authorize the connection so Dataddo can write files to SFTP server:
| Field | Description |
|---|---|
| A name for this authorizer in Dataddo | A label so you can recognize the connection later. |
| Server IP or Hostname | Public IP or Hostname of your SFTP Server. |
| Username | Username for authentication. |
| Password | Password for authentication. |
| Port | Port to connect to SFTP. The default value is 22. |
| Timeout | Timeout Default: 90. |
| Use Certificate | Use Certificate |
| Certificate | Certificate for certificate-based authentication. You can upload or generate certificate in Security settings. Shown when useCertificate is ``. |
Dataddo validates the connection when you save it.
Create a SFTP server Destination
Go to Destinations, click Create Destination, and select SFTP server. Give the destination a name, choose the authorizer, then set:
| Field | Description |
|---|---|
| Path | Enter or select the directory where the data file will be created. Shown when oAuthId is ``. |
Click Save.
Supported File Formats
Each flow run writes the data as a file in the format you pick.
| Format | Description |
|---|---|
csv |
SFTP server writes the data as CSV files. |
json |
SFTP server writes the data as JSON files. |
jsonl |
SFTP server writes the data as JSONL files. |
parquet |
SFTP server writes the data as PARQUET files. |
For CSV you can set the delimiter, header row, and date formatting. For Parquet, JSON, and JSONL you can set the timestamp unit.
File Naming
You can also build a custom filename with these placeholders (see Dynamic File Naming Patterns for the full list):
{{objectLabel}}and{{objectId}}: the flow name and id.{{today}}and{{yesterday}}: the run date.{{dateRangeStart}}and{{dateRangeEnd}}: the bounds of the flow's date range.- Date-range expressions such as
{{1d1}}(yesterday) or{{90d1}}(the last 90 days through yesterday). - Add a date format after a
|, for example{{today|Ymd}}gives20201231and{{1d1|Y-m-d}}gives2020-12-31.
File Partitioning
File partitioning splits a large dataset into smaller files based on a criterion such as date, which improves how a data lake organizes and queries the data (see Data Lake Ingestion). In Dataddo you partition by putting date variables from File Naming into the file name, so each flow run writes its own dated file.
For example, events_{{1d1|Y-m-d}}.parquet writes one Parquet file per day, so a lake engine such as SFTP server can read the set of files as a date-partitioned dataset. Pick a file format like Parquet or CSV that your lake reads, and schedule the flow to match the partition period (for example daily for a daily date token).
Write Modes
Each flow run writes a file at the resolved name. The default is truncate_insert: insert keeps writing new files, and truncate_insert replaces the file at the same name.
How Data Is Delivered
Every run produces one file at the resolved name. A date-stamped name accumulates a new snapshot file per run, which is the pattern described in Data Lake Ingestion.
How to Create a Flow to SFTP server
- Go to Flows and click Create Flow.
- Add one or more sources.
- Add SFTP server as the destination and pick the authorizer.
- Choose the file format and the file name.
- Set the schedule and click Save.
Troubleshooting
Cannot connect to SFTP server
Dataddo cannot reach SFTP server. Check the credentials, path, and permissions in the authorizer and destination, and make sure SFTP server is reachable by Dataddo.
File is not created
The account used by Dataddo lacks write permission on the target path. Grant write access to the folder or bucket and restart the flow.