- 5 Minutes to read
- 5 Minutes to read
SFTP, or Secure File Transfer Protocol, is a network protocol used for securely transferring files between a client and a server. An SFTP server is a server that supports this protocol, enabling secure and encrypted file transfers over a network, making it a safer and more protected alternative to traditional FTP for data exchange.
Authorize Connection to SFTP Server
Please keep in mind that this guide provides general instructions, and your individual setup might need a different approach or some extra steps. Always take your organization's security guidelines into account when you're making these changes. Additionally, it's a good idea to seek assistance from a database administrator or an IT professional to make sure everything is set up correctly.
Enable Port Forwarding
Enable port forwarding to make sure your server is accessible from the Internet by following these steps.
- Assign a static IP to your server on your local network. This makes sure that the server's address remains constant, so other devices and services always know where to find it.
- Windows: You can set a static IP from the network settings.
- Linux: This is typically done by editing network configuration files, which vary by distribution (e.g., /etc/network/interfaces or /etc/netplan/ directory for Ubuntu).
- Configure port forwarding on your router: Log in to your router's web interface and look for a section called Port Forwarding, Virtual Server, or something similar. Create a new port forwarding rule:
- Service or Application: Name it e.g. SFTP Server.
- Internal IP: Enter the static IP address of your server.
- Port Range: For SFTP, the standard port is 22.
- Protocol: Usually TCP for FTP and SFTP.
- External IP: Unless you are restricting access to specific external IPs, you can leave this set to all or the equivalent option.
- Save your new rule.
To create a user with required permissions for SFPT, please follow these steps.
- Right-click This PC or My Computer, select Manage and open the Computer Management tool.
- Navigate to Local Users and Groups and select Users.
- Right-click Users and proceed with creating a new user. Use something like
DataddoUserand create a strong password.
Configure Access Permissions
- Locate your folder, right-click in and select Properties.
- Navigate to the Security tab and click on Edit.
- Click on Add to add your user or a group. Enter the username and verify.
- With the user selected in the Group or user names section, in the Permissions for [
DataddoUser] section, check the permissions you want to assign to this user:
- Full control
- Read & execute
- List folder contents
- Click on Apply to save your changes.
Allow Network Connection
For this destination, whitelisting IP addresses is not available in the database system. Instead, you can configure the network access at the firewall level. Here are the general steps:
- Identify the firewall that's protecting your service. This could be a Windows or Linux firewall running on the server itself, or it could be a network firewall.
- Open the firewall's configuration settings. This process will vary depending on the type of your firewall.
- Look for the setting that allows you to define inbound rules or policies.
- Create a new rule or policy that allows inbound traffic on the port that the service is using (the default is 22) from the Dataddo IP addresses.
- Save your changes and test the connection from Dataddo to make sure everything is working as intended.
- On the Authorizers page, click on Authorize New Service and select FTP server.
- You will be asked to fill the following fields
- Server IP or Host name: The address of your SFTP server.
- Username: The username for the SFTP account.
- Password: The password for the SFTP account.
- Port: The port your SFTP server is running on (default value is 22).
- [Optional] Configure the connection via SSH tunnel. To do so, please follow these steps.
- Click on Save.
Create a New SFTP Destination
- Under the Destinations tab, click on the Create Destination button and select the destination from the list.
- Select your account from the drop-down menu.
- Fill in the Path to your directory.
- Name your destination and click on Save to create your destination.
Click on Add new Authorizer in drop-down menu during authorizer selection and follow the on-screen prompts. You can also go to the Authorizers tab and click on Add New Service.
Creating a Flow to FTP Server
- Navigate to Flows and click on Create Flow.
- Click on Connect Your Data to add your sources.
- Click on Connect Your Data Destination to add the destination.
- Choose the write mode and fill in the other required information.
- Check the Data Preview to see if your configuration is correct.
- Name your flow, and click on Create Flow to finish the setup.
File partitioning refers to the practice of dividing large datasets into smaller, more manageable segments or partitions based on specific criteria, such as values in a particular column or range of values. Each partition contains a subset of the data that shares common attributes or characteristics. File partitioning is commonly used to improve data organization, query performance, and data management.
Dataddo supports file partitioning during flow creation. If you use, for example,
Dataddo will create a file every day, e.g. file_20xx0101, file_20xx0102 etc.
Configuring User Permissions in Linux
To create a user with the necessary permissions for SFTP in Linux, follow these steps.
- Connect to your server: Access your server's command line interface. This might be directly or via SSH, using a host address and port that your server operates on.
- Create a new user: This can typically be done using the useradd or adduser command followed by the desired username. Follow the prompts to set a strong password and, optionally, fill in any additional user information. For instance:
sudo adduser new_user
- Create a directory for SFTP access: If it doesn’t already exist, create a directory that you want the new user to access via SFTP. For example:
sudo mkdir -p /home/new_user/file_name
- Set the owner and permissions for the directory: You need to set the appropriate ownership and permissions for the directories and files that the user should access. For example:
sudo chown new_user:new_user /home/new_user/file_name sudo chmod 700 /home/new_user/file_name
- Configure SFTP settings in the SSHD config file: Open the SSH daemon's configuration file in a text editor. This is usually located at
/etc/ssh/sshd_config. You might add or modify a section like the following:
Match User new_user ChrootDirectory /home/new_user ForceCommand internal-sftp AllowTCPForwarding no X11Forwarding no
- Restart the SSH service: Apply the changes by restarting SSH.