Fixed-Schema Connectors

Fixed-schema connectors provide a set of dataset templates which typically represents an definition of a call to API endpoint and transformation pipeline expression. The examples of such connectors is Shopify, ExactOnline, Xero, Stripe or Semrush.

Prerequisites

• You have the link to API endpoint documentation.
• You have successfuly created an Authorizer to the desired service.

Implementation

Fixed-schema connectors require the following workflow:

1. Call the /connectors/{connectorId}/actions/authorization endpoint for obtain the list of active authorizers. If none is available create a new one. ID of the authorizer is passed as oAuthId parameter in the next stages.
2. Call the /connectors/{connectorId}/templates endpoint to acquire the list of available templates. Use ID of the template to obtain template details by calling /connectors/{connectorId}/templates/{id}. Each template consist of request object with url, header, body, method, transformation and emitter attributes. You will need those in the following steps.
3. Check API endpoint documentation for availability of /actions/* endpoints. This indicate that there are additional parameters that needs to be supplied.
4. Check attributes in the request object (i.e. url, header, transformation) if containing placeholders with double curly brackets (e.g. {{subdomain}}). If, so this indicates that such param need to be supplied in parameters attribute.
5. As an example, if request.url has the value https://{{subdomain}}.shopify.com you will need to pass the attribute subdomain with your value as part of the parameters object.
6. Send the full payload to the /preview endpoint to test the functionality of the payload. If successful, you can create the source by calling /create-source endpoint.

Payload schema

Use this schema when calling /preview and /create-source endpoints for testing the payload and creating the source. The payload structure is common for all fixed-schema connectors.

{
    "connectorId":"CONNECTOR_ID",
    "templateId":"TEMPLATE_ID",
    "label":"My Test",
    "type":"CONNECTOR_ID",
    "oAuthId":"1234",
    "request": {
        "url":"",
        "header":[],
        "method":"GET",
        "body":"",
        "transformation":"[]",
        "emitter":"[]"
    },
    "parameters":{},
    "syncFrequency":"custom",
    "hour":"3",
    "minute":"25",
    "dayOfWeek":"*",
    "dayOfMonth":"*",
    "month":"*",
    "timezone":"UTC",
    "strategy":"clean"
}