The BigTime connector lets you extract data from BigTime into Dataddo and sync it to any dashboard, database, or data warehouse.
Refer to our website for the list of available metrics and attributes you can extract from BigTime.
If you have multiple BigTime accounts and would like to extract the same data from all of them, it's possible through multi-account extraction. Contact our Solutions team to enable this feature.
Data Coverage
BigTime exposes the following datasets. Each dataset maps to a table you can extract. Example fields are a representative sample; each dataset returns more columns.
| Dataset | Description | Example fields | Date range |
|---|---|---|---|
| Reports | Sync data from a BigTime report. Since this endpoint returns all-time totals, configure your sync to use a replace storage strategy. | dynamic (from your account) | No |
How Data Extraction Works
None of this connector's datasets use a date range. Every run pulls all currently available data.
Metadata Columns
When you create a source, you can add these Dataddo metadata columns to the extracted data:
- dataddo_hash - a fingerprint built from each record's key fields. It works as a natural key, so it is ideal for upserts (updating existing rows in your destination instead of creating duplicates).
- dataddo_extraction_timestamp - the date and time the row was extracted. Use it to track how records change over time, for example to build slowly changing dimensions.
How to Create a BigTime Data Source
Creating a data source takes you through six steps, shown in the progress bar at the top of the wizard. Each step is explained below.
1. Pick the connector
On the Sources page, click Create Source, then select the connector from the catalog. Use the search bar or the category tabs if you do not see it right away. You can rename the source at any time using the pencil icon next to its name.
2. Select the dataset
A dataset defines the shape of your data: which fields you get and how they relate. Select the dataset you want; you can still fine-tune the exact fields later.
- Each dataset has a short description of what it contains. Use the search box to find a dataset, attribute, or metric by name.
- The panel on the right previews the selected dataset's fields. For each field you can see its data type, whether it holds sensitive data (personal fields such as name or email are flagged), and which other datasets it links to, so you can see how the datasets relate.
3. Choose the account
This step selects what Dataddo reads from.
- Authorizer: Select an account you have already authorized from the drop-down. If you have none yet, choose Add new account and follow the prompts. If no authorizer is selected, Dataddo asks you to authorize before you continue.
- What to extract from: Select the exact entity you want to pull data from. Depending on the service this may be labelled an account, property, profile, workspace, or similar, sometimes with a sub-level to choose as well.
- Multiple accounts: To pull the same data from every entity you can access, turn on Automatically collect data from all .... This is multi-account extraction. Leave it off to choose them by hand.
4. Refine the attributes and metrics
The dataset already sets the structure. Here you fine-tune it: tick or untick the specific attributes and metrics you want to keep, and use the search box to find a field quickly. Click Test on Sample Data at any point to preview the result before you continue.
5. Add metadata columns (optional)
Two optional columns help your destination handle the data.
- Dataddo Hash (Include Row Hash): a fingerprint built from the columns you pick. It works as a natural key, so your destination can deduplicate rows and run upserts instead of creating duplicates. Turn it on, then select the columns that uniquely identify a row.
- Dataddo Extraction Timestamp: the time each row was extracted. Use it to watermark the data, for example to build slowly changing dimensions or to track when a value last changed.
6. Set the schedule
Decide how often Dataddo runs the extraction.
- Frequency: how often the pipeline runs, for example daily. Click Show advanced settings to also set the exact hour and minute (UTC).
- Date range: the relative window each run extracts, for example "Yesterday". The window moves forward on every run.
- Historical data: a new source starts from the current window. To load older data, run a full data re-sync after the source is created.
- Allow Empty Data Extractions: when on, a run that returns no data records zero rows instead of failing. Turn it on if the source can legitimately have periods with no data.
Click Save. Your data source is ready.
Related Articles
Now that you have successfully created a data source, see how you can connect your data to a dashboarding app or a data storage.
Sending Data to Dashboarding Apps
Sending Data to Data Storages
Other Resources