Documentation Index

Fetch the complete documentation index at: https://docs.dataddo.com/llms.txt

Use this file to discover all available pages before exploring further.

Google Cloud Storage

Prev Next

Google Cloud Storage is Google Cloud's object storage service. The Dataddo destination writes each flow run as a file into a bucket and folder you choose, in the format you select.

Authorize Connection to Google Cloud Storage

You can authorize Google Cloud Storage in two ways. Pick the one that fits your setup.

Google Cloud Storage (OAuth)

Sign in with your Google account and grant Dataddo access. Dataddo stores the resulting OAuth token, so you do not enter any keys by hand.

  1. Go to Authorizers > Authorize New Service.
  2. Select Google Cloud Storage.
  3. Complete the Google sign-in and consent screen.
  4. The authorizer is saved once Google returns a valid token.

Google Cloud Storage (Service account)

Use a service account when you cannot use interactive sign-in, for example for a shared or automated setup.

  1. In the Google Cloud Console, open IAM & Admin > Service Accounts and create a service account.
  2. Create and download a JSON key file for that service account.
  3. In Dataddo, open Certificates and Tokens under settings and upload the JSON file.
  4. Go to Authorizers > Authorize New Service > Google Cloud Storage (Service account) and fill in the fields below.
Field Description
Label A name for this authorizer in Dataddo.
Configuration file The JSON key file you uploaded under Certificates and Tokens.

Grant the account read and write access to the target bucket (for example the Storage Object Admin role) so Dataddo can create and overwrite files.

Create a Google Cloud Storage Destination

Go to Destinations > Create Destination > Google Cloud Storage, then choose the authorizer you created and set the fields below.

Field Description
Destination Name A name for this destination in Dataddo.
Project ID The Google Cloud project that owns the bucket. The list loads from your account after you select the authorizer. If no project appears, reauthorize the authorizer.
Bucket The bucket where files are written. The list loads from the selected project.
Path The folder inside the bucket, for example path/to/my/directory/. Use / for the bucket root. The path must end with a slash.

Supported File Formats

Choose the output format in the flow. Each format exposes its own options.

Format Options
CSV CSV Delimiter (default ;), Include Header (default on), Null Dates as Empty to write null dates as empty strings instead of 1970-01-01 (default off), and Custom DateTime Format (default YMDHIS).
JSON JSON timestamp unit (default nanoseconds).
JSONL JSONL timestamp unit (default nanoseconds).
Parquet Parquet timestamp unit (default microseconds). Use microseconds for Databricks compatibility.
XML No extra options.
XLSX No extra options.

File Naming

The File Name field controls the name of each written file. Pick a preset or choose Custom Filename to enter your own pattern. The default is {{objectLabel}}-{{today|Ymd}}.

Preset patterns:

  • {{objectLabel}}-{{today|Ymd}}
  • {{objectLabel}}-{{today|YmdHis}}
  • {{objectLabel}}
  • {{today|Ymd}}
  • {{today|YmdHis}}
  • {{1d1|Y-m-d}}

You can build custom names from these placeholders:

Placeholder Meaning
{{objectLabel}} The name of the flow.
{{today\|Ymd}} The run date. The text after \| is a PHP date format, so Ymd gives 20260906 and YmdHis adds the time.
{{1d1}} A date range token that resolves to a past date, for example {{1d1\|Y-m-d}} for yesterday.

Write Modes

Write mode decides what happens to the target file on each run. Availability of a mode is resolved by Dataddo's writer at flow time, so the exact list can depend on the destination state. The flow default is insert.

Mode Behavior for files
insert Writes a new file for the run at the resolved file name.
truncate_insert Replaces the file at the resolved name with the current data.

When the file name contains a date placeholder, each run resolves to a different name and the bucket accumulates one snapshot file per run. See Data Lake Ingestion for how snapshot files build up over time.

How Data Is Delivered

Every flow run produces one file at the resolved path and name. A static file name is overwritten each run. A date-stamped name creates a new file each run and keeps the history in the bucket.

How to Create a Flow to Google Cloud Storage

  1. Go to Flows > Create Flow.
  2. Add one or more sources.
  3. Add the Google Cloud Storage destination.
  4. Choose the file format and set its options.
  5. Set the file name pattern.
  6. Choose the write mode.
  7. Set the schedule and save.

Troubleshooting

  • No project or bucket in the list. Reauthorize the Google Cloud Storage authorizer, then reselect the project so the bucket list can load.
  • Path rejected. The path must match path/to/my/directory/ and end with a slash. Use / for the bucket root.
  • Access denied on write. Confirm the authorized account or service account has write access to the bucket.

Related Articles