Google Search Console

Google Search Console is a free web service that allows website owners and webmasters to monitor and manage how their site appears in Google search results. It provides valuable insights into search performance, indexing status, and helps identify and resolve issues that might affect a website's visibility in Google's search engine.

Refer to our website for the list of metrics and attributes available in Dataddo.

Refer to Google's official documentation to see all available endpoints from the Google Search Console API.

Authentication Methods

Google Search Console supports more than one way to connect. Pick one when you create the authorizer in Dataddo:

• Google Search Console - sign in with Google Search Console and approve access (recommended).
• Google Search Console Custom - use your own app credentials (for advanced setups).

Authorize Connection to Google Search Console

To authorize this service, use OAuth 2.0 to share specific data with an application while keeping usernames, passwords, and other information private.

1. On the Authorizers page, click on Authorize New Service and select your service.
2. Follow the prompts to grant Dataddo the necessary permissions to access and retrieve your data.
3. [Optional] Once your authorizer is created, click on it to change the label for easier identification.

Ensure that the account you're granting access to is a Site Owner or a Site Full User.

For more information, see our article on authorizers.

Data Coverage

Google Search Console exposes the following datasets. Each dataset maps to a table you can extract. Example fields are a representative sample; each dataset returns more columns.

How Data Extraction Works

What each extraction pulls depends only on whether a dataset supports a date range (see the Date range column above):

• Date range supported (Yes): the source reads a relative window (for example "last 7 days"), and that window slides forward with the current date. Every run re-reads the window, so a range of "1 day ago" always pulls the previous day (D-1). Each run replaces the window's data rather than adding older history. To load records from before the window, run a full data re-sync with a wider range. See Data Backfilling.
• No date range (No): every run pulls all currently available data.

Set the relative date range when you create the source.

Metadata Columns

When you create a source, you can add these Dataddo metadata columns to the extracted data:

• dataddo_hash - a fingerprint built from each record's key fields. It works as a natural key, so it is ideal for upserts (updating existing rows in your destination instead of creating duplicates).
• dataddo_extraction_timestamp - the date and time the row was extracted. Use it to track how records change over time, for example to build slowly changing dimensions.

How to Create a Google Search Console Source

Creating a data source takes you through six steps, shown in the progress bar at the top of the wizard. Each step is explained below.

1. Pick the connector

On the Sources page, click Create Source, then select the connector from the catalog. Use the search bar or the category tabs if you do not see it right away. You can rename the source at any time using the pencil icon next to its name.

2. Select the dataset

A dataset defines the shape of your data: which fields you get and how they relate. Select the dataset you want; you can still fine-tune the exact fields later.

• Each dataset has a short description of what it contains. Use the search box to find a dataset, attribute, or metric by name.
• The panel on the right previews the selected dataset's fields. For each field you can see its data type, whether it holds sensitive data (personal fields such as name or email are flagged), and which other datasets it links to, so you can see how the datasets relate.

3. Choose the account

This step selects what Dataddo reads from.

• Authorizer: Select an account you have already authorized from the drop-down. If you have none yet, choose Add new account and follow the prompts. If no authorizer is selected, Dataddo asks you to authorize before you continue.
• What to extract from: Select the exact entity you want to pull data from. Depending on the service this may be labelled an account, property, profile, workspace, or similar, sometimes with a sub-level to choose as well.
• Multiple accounts: To pull the same data from every entity you can access, turn on Automatically collect data from all .... This is multi-account extraction. Leave it off to choose them by hand.

4. Refine the attributes and metrics

The dataset already sets the structure. Here you fine-tune it: tick or untick the specific attributes and metrics you want to keep, and use the search box to find a field quickly. Click Test on Sample Data at any point to preview the result before you continue.

5. Add metadata columns (optional)

Two optional columns help your destination handle the data.

• Dataddo Hash (Include Row Hash): a fingerprint built from the columns you pick. It works as a natural key, so your destination can deduplicate rows and run upserts instead of creating duplicates. Turn it on, then select the columns that uniquely identify a row.
• Dataddo Extraction Timestamp: the time each row was extracted. Use it to watermark the data, for example to build slowly changing dimensions or to track when a value last changed.

6. Set the schedule

Decide how often Dataddo runs the extraction.

• Frequency: how often the pipeline runs, for example daily. Click Show advanced settings to also set the exact hour and minute (UTC).
• Date range: the relative window each run extracts, for example "Yesterday". The window moves forward on every run.
• Historical data: a new source starts from the current window. To load older data, run a full data re-sync after the source is created.
• Allow Empty Data Extractions: when on, a run that returns no data records zero rows instead of failing. Turn it on if the source can legitimately have periods with no data.

Click Save. Your data source is ready.

Limitations

Historical Data Limitations

The Google Search Console API allows you to extract data up to 16 months back.

Troubleshooting

Data Preview Unavailable

No data preview when you click on Test Data might be caused by an issue with your source configuration. The most common causes are:

• Date range: Try a smaller date range. You can load the rest of your data afterward via manual data load.
• Insufficient permissions: Please make sure your authorized account has at least admin-level permissions.

Aggregated Data by Property vs. by Page

When you aggregate data in Google Search Console, you have the option to do it either by property (all domains inside one URL) or by page.

Aggregation by Property
When you aggregate by property, all domains inside a specific URL will be combined under one entry. E.g. dataddo.com/connectors and dataddo.com/platform will both be counted under dataddo.com.

Aggregation by Page
When aggregating by page, each unique URL will be counted once, even if they are pointing to the same page.

While the numbers will match those in your GSC UI, aggregated totals by property and page will not match which may appear as data discrepancy.

For more detailed information, please see Google's official documentation.

Aggregated Data Discrepancies

If you aggregate your impressions or clicks data by countries or devices, the data will not have any discrepancies.

However, if you aggregate values by query and/or page, there will potentially be an infinite number of rows due to the very large or infinite number of search queries. Due to data sampling, as much data as possible will be shown but it won't be all data (all search queries).

This means that while impressions or clicks per query will be the same, the aggregated values for data in your GSC and data extracted by Dataddo will be different. (Trends should be the same.) The exact same thing applies for the page dimension.

Solution
To avoid data discrepancies, use only date, country, or device dimensions or their combination for totals.
Also when selecting your dimensions, for less data discrepancies, use as few as possible.

Cannot Retrieve Fresh Data

Google takes between 24 - 72 hours to process data which is why it’s possible to get data only from 3 days ago or older. For Google Search Console specifically, change your date range expression to 4d4 or older.

Related Articles

Now that you have successfully created a data source, see how you can connect your data to a dashboarding app or a data storage.

Sending Data to Dashboarding Apps

• Simple Data Integration to Dashboards
• Data Backfilling to Dashboarding Apps

Sending Data to Data Storages

• Batch Ingestion to Data Warehouses
• Data Backfilling to Storages

Other Resources

• Troubleshooting
• Extraction Logs
• Data Duplication