How to connect Firebolt

Getting your data to Firebolt is an easy task with Dataddo. If you already have your Firebolt account connected to Dataddo, and you only need instructions on how to connect it to a Data Flow, you can skip Step 1 and start from Step 2 right away.

Table of content:

Firebolt overview

About Firebolt

Firebolt is a cloud-based data storage and analytics service built over AWS, suitable for massive data sets. you can connect your data to Firebolft quickly with Dataddo.

Prerequisites

In order to connect your data to Firebolt, you need to have:

  • An active Firebolt account
  • The username and password of your Firebolt account
  • A General Purpose engine attached to your database. A general-purpose engine offers read and write ability and better performance compared to other engine types.
  • A Firebolt database to which you want to load data.

Tables

When you create a table in Firebolt to run queries over, you must specify whether it’s a FACT or a DIMENSION table. Firebolt handles these table types differently to optimize query performance in general and join operations in particular.

  • Fact tables are always sharded across engine nodes. Each node stores part of the table. Use fact tables for your larger and most granular (transaction) tables.
  • Dimension tables are replicated in each engine node. Use dimension tables for smaller data sets that are typically more descriptive and are frequently joined with fact tables. When performing joins, the local shard of a fact table on each node is joined with the local copy of the dimension table.

NOTE:

With the Dataddo app, you can create FACT tables. If you need a DIMENSION table, first you need to create the table in the Firebolt (go to Databases > Tables > Add new table) and then send the data to the particular table

If your table does not fit in either of the traditional fact/dimension definition, then it is recommended to define very large tables as fact, and smaller tables as dimensions.

Primary indexes

Firebolt incorporates several building blocks (indexes) that enable accelerated query response times.

The primary indexes are the main sort key of the table. They will be used as the main tool for data pruning and data distribution.

By using the primary index, Firebolt will read ranges of data from each F3 file. This will reduce the I/O needed to be read, and increase performance dramatically.

How to choose a primary index? Read more in the official documentation.

Limitations

  • Once a table is created in the Firebolt database, it cannot be altered, meaning there is no option to add or remove columns. Firebolt does not support this.

  • Firebolt supports only Insert (append new data) write mode.

How to connect Firebolt

Step 1 - Create a Destination

   
Number 1

Sign in to your Dataddo account and click on Destinations at the top of the page.

Number 2

Click on Create Destination in the top right corner.Data Destination - Create Destination

Number 3 From the list of destinations, choose Firebolt. You can type the connector's name into the search bar to find it faster.
Data destinations - all
Number 4

Select the name of the destination, Firebolt user name, Password, Database and Region. Click on Save. Firebolt- step 4

Step 2 - How to create Firebolt Data Flow

   
Number 1

Sign in to your Dataddo account and click on Flows at the top of the page. 

Number 2

Click on Create flow in the top right corner.My Flows - Create Flow

Number 3 Click on Add Source to connect it to the destination. You can type the connector's name into the search bar to find it faster.
My Flows - New Flow
Number 4 Once you selected your data source, click on Add Destination.
Firebolt- Flow 4
Number 5

 

From the list of destinations, under the tab Data Storages, select Firebolt. You can also type the name of the source into the search bar.Firebolt- Flow 5
Number 6 Configure the destination by filling out necessary fields, choose the name of the flow and click on Save flow.
Firebolt- Flow 6 configuration

NOTE: Remember to set the name time for the flow with a little delay after the synchronization of your source to let the data load.

Number 7

 

You can see what data will be sent to the destination, click on Preview.

NOTE: The preview is limited and not the entire table will be displayed.
Google Sheets- Flow 7

How to configure Firebolt Data Flow

In order to send the data to Firebolt.io, the database table must exist.
On the flow first run, Dataddo automatically creates the database table according to your dataset and configuration.

   
Number 1

Sign in to your Dataddo account and click on Flows at the top of the page. Click on the three dots next to the flow and choose Config. Firebolt - configure 1

Number 2

The window pops up with the configuration details.Firebolt - configure 2


Need assistance?

Feel free to contact us or create a ticket and we will help you set up the Data Source.