S3

Amazon S3 (Simple Storage Service) is a scalable cloud storage service provided by Amazon Web Services (AWS). It offers a way to store and retrieve data, such as files, images, videos, and backups, in a highly durable and easily accessible manner, making it a foundational component for various cloud-based applications and services.

Prerequisites

Authorize the Connection to Amazon S3

In AWS Portal

Create S3 Bucket

1. Sign in to the AWS Management Portal and navigate to S3 Console.
2. Click on +Create bucket. Name your bucket and choose AWS Region. You will need to provide this information to Dataddo.
3. Click on the Create button to finalize the process.

Configure the Access Permissions

1. Navigate to the IAM service, click on Users and continue with Add User.
2. Name your user and for the access type, select Programmatic access.
3. In the Set permissions step, choose Attach existing policies directly. Attach a policy that grants the necessary S3 permissions. This could be
   • The AmazonS3FullAccess policy, which grants full access to all S3 resources, or
   • A custom policy that only grants access to the necessary bucket and actions.
4. Save the IAM user's credentials. You will be given an access key and a secret key. These credentials will be needed for configuring the connection to your S3 bucket in Dataddo.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl",
        "s3:GetObject",
        "s3:GetObjectAcl",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::your-bucket-name/*"
    }
  ]
}

In Dataddo

1. In the Authorizers tab, click on Authorize New Service and select S3.
2. You will be asked to fill the following fields
   1. Bucket: Provide the identifier of S3 bucket you want to use for reading or writing the data.
   2. Region: Region of the S3 bucket.
   3. Key: Provide your AWS Access Key.
   4. Secret: Provide your AWS Secret Key.
3. Click on Save.

Create a New S3 Destination

1. Under the Destinations tab, click on the Create Destination button and select the destination from the list.
2. Select your account from the drop-down menu.
3. Fill in the Path. Use the name of the folder in your Container + slash (e.g. "database/" ).
4. Name your destination and click on Save to create your destination.

Creating a Flow to S3 Storage

file_{{1d1|Ymd}}