Configuration

  • Updated

List of flows in ErrorsIO-Snowflake:

  1. DF01: Integrator IO Integrations to Snowflake Integrations Add
  2. DF02: Integrator IO Flows to Snowflake Flows Add
  3. DF03: Integrator IO Exports to Snowflake Exports Add
  4. DF04: Integrator IO Imports to Snowflake Imports Add
  5. DF05: Integrator Open Errors to Snowflake Errors Add
  6. DF06: Update Integrator IO Integration Status to Snowflake Integrations
  7. DF07: Update Integrator IO Integration Flows to Snowflake Flows
  8. DF08: Update Integrator IO Integration Exports/Imports to Snowflake Exports and Imports

DF01: Integrator IO Integrations to Snowflake Integrations Add:

Technical Design:

This flow exports integrations from the integrator.io and imports them into Snowflake integrations table on schedule basis.

Introduction:

Sync integrator.io integrations to Snowflake: This integration flow syncs integrations from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from integrator.io and Import into Snowflake.
  • IO Resource: Integration
  • IO API Name: integrations
  • Operations: Add (or Update)

Setup Instructions:

create or replace TABLE {database_name}.{schema_name}.INTEGRATIONS (
ID NUMBER(38,0) NOT NULL autoincrement,
_INTEGRATION_ID VARCHAR(1000) NOT NULL,
_INTEGRATION_NAME VARCHAR(1000),
_CONNECTOR_ID VARCHAR(1000),
_ACCOUNT_ID VARCHAR(1000),
_ACCOUNT_NAME VARCHAR(1000),
CREATED_AT TIMESTAMP_NTZ(9),
_RECORD_ADDED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
_RECORD_UPDATED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
STATUS BOOLEAN,
IS_SANDBOX BOOLEAN DEFAULT FALSE COMMENT '0 for production. 1 for sandbox.',
constraint PK_INTEGRATIONS primary key (_INTEGRATION_ID)
);

Prerequisites:

  • Create the table INTEGRATIONS in Snowflake before running the flow

DF02: Integrator IO Flows to Snowflake Flows Add:

Technical Design:

This flow exports flows from the integrator.io and imports them into Snowflake flows table on schedule basis.

Introduction:

Sync integrator.io flows to Snowflake: This integration flow syncs flows from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from integrator.io and Import into Snowflake.
  • IO Resource: Flows
  • IO API Name: flows
  • Operations: Add (or Update)

Setup Instructions:

  • Create database table in Snowflake to save the integration details
create or replace TABLE {database_name}.{schema_name}.INTEGRATION_FLOWS (
ID NUMBER(38,0) NOT NULL autoincrement,
_FLOW_ID VARCHAR(1000) NOT NULL,
_FLOW_NAME VARCHAR(1000),
_INTEGRATION_ID VARCHAR(1000) NOT NULL,
_INTEGRATION_NAME VARCHAR(1000),
CREATED_AT TIMESTAMP_NTZ(9),
STATUS BOOLEAN,
_FLOW_TYPE VARCHAR(1000) NOT NULL,
_RECORD_ADDED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
_RECORD_UPDATED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
primary key (_FLOW_ID)
);

Prerequisites:

  • Create the table INTEGRATION_FLOWS in Snowflake before running the flow

DF03: Integrator IO Exports to Snowflake Exports Add:

Technical Design:

This flow exports ‘exports’ from the integrator.io and imports them into Snowflake ‘INTEGRATION_FLOW_EXPORTS_AND_IMPORTS’ table on a schedule basis.

Introduction:

Sync integrator.io exports to Snowflake: This integration flow syncs exports from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from integrator.io and Import into Snowflake.
  • IO Resource: Exports
  • IO API Name: exports
  • Operations: Add (or Update)

Setup Instructions:

  • Create database table in Snowflake to save the integration details
create or replace TABLE {database_name}.{schema_name}.INTEGRATION_FLOW_EXPORTS_AND_IMPORTS (
ID NUMBER(38,0) NOT NULL autoincrement,
_RECORD_ID VARCHAR(1000),
_RECORD_NAME VARCHAR(1000),
_FLOW_ID VARCHAR(1000),
_FLOW_NAME VARCHAR(1000),
_INTEGRATION_ID VARCHAR(1000),
_INTEGRATION_NAME VARCHAR(1000),
_AUTO_INC_FLOW_ID NUMBER(38,0) NOT NULL,
TYPE VARCHAR(255) NOT NULL DEFAULT 'exports',
CREATED_AT TIMESTAMP_NTZ(9),
STATUS BOOLEAN,
_RECORD_ADDED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
_RECORD_UPDATED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
unique (_RECORD_ID, _FLOW_ID),
primary key (ID),
constraint FKEY_1 foreign key (_FLOW_ID) references IO_ERROR_MANAGEMENT.IO_ERROR_MANAGEMENT.INTEGRATION_FLOWS(_FLOW_ID)
);

Prerequisites:

  • Create the table INTEGRATION_FLOW_EXPORTS_AND_IMPORTS in Snowflake before running the flow

DF04: Integrator IO Imports to Snowflake Imports Add:

Technical Design:

This flow exports ‘imports’ from the integrator.io and imports them into Snowflake ‘INTEGRATION_FLOW_EXPORTS_AND_IMPORTS’ table on a schedule basis.

Introduction:

Sync integrator.io imports to Snowflake: This integration flow syncs imports from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from integrator.io and Import into Snowflake.
  • IO Resource: Imports
  • IO API Name: imports
  • Operations: Add (or Update)

Setup Instructions:

  • Make sure INTEGRATION_FLOW_EXPORTS_AND_IMPORTS table is created, if not please create as mentioned in DF03

Prerequisites:

  • Create the table INTEGRATION_FLOW_EXPORTS_AND_IMPORTS in Snowflake before running the flow

DF05: Integrator Open Errors to Snowflake Errors Add:

Technical Design:

This flow exports open export errors from the integrator.io and imports them into the Snowflake errors table on schedule basis.

Introduction:

Sync integrator.io errors to Snowflake: This integration flow syncs errors from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from integrator.io and Import into Snowflake.
  • IO Resource: Errors
  • IO API Name: errors
  • Operations: Add (or Update)

Setup Instructions:

  • Create database table in Snowflake to save the integration details
create or replace TABLE {database_name}.{schema_name}.INTEGRATION_ERRORS (
ID NUMBER(38,0) NOT NULL autoincrement,
MESSAGE VARCHAR(1000),
_ERROR_ID VARCHAR(1000) NOT NULL,
_CODE VARCHAR(1000),
_SOURCE VARCHAR(1000),
_FLOW_JOB_ID VARCHAR(1000),
_AUTO_INC_RECORD_ID NUMBER(38,0) NOT NULL,
_ERROR_DATE TIMESTAMP_NTZ(9) NOT NULL,
_RECORD_ADDED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
_RECORD_UPDATED_DATE TIMESTAMP_NTZ(9) DEFAULT CAST(CONVERT_TIMEZONE('UTC', CAST(CURRENT_TIMESTAMP() AS TIMESTAMP_TZ(9))) AS TIMESTAMP_NTZ(9)),
primary key (_ERROR_ID),
constraint FKEY_1 foreign key (_AUTO_INC_RECORD_ID) references IO_ERROR_MANAGEMENT.CUSTOMER_IO_ERROR_MANAGEMENT.INTEGRATION_FLOW_EXPORTS_AND_IMPORTS(ID)
);

Prerequisites:

  • Create the table INTEGRATION_ERRORS in Snowflake before running the flow

DF06: Update Integrator IO Integration Status to Snowflake Integrations:

Technical Design:

This flow exports integration details from Snowflake and verifies if it is still available in IO. If integration is deleted and not available in IO, it gets marked as inactive in Snowflake.

Introduction:

Sync integrator.io integrations status to Snowflake: This integration flow syncs integration status from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from Snowflake, Lookup in integrator.io and import into Snowflake.
  • IO Resource: Integration
  • IO API Name: integrations
  • Operations: Update

Prerequisites:

  • Create the table INTEGRATIONS in Snowflake before running the flow

DF07: Update Integrator IO Integration Flows to Snowflake Flows:

Technical Design:

This flow exports flow details from Snowflake and verifies if it is still available in IO. If flow is deleted and not available in IO, it gets marked as inactive in Snowflake.

Introduction:

Sync integrator.io flow status to Snowflake: This integration flow syncs integration status from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from Snowflake, Lookup in integrator.io and import into Snowflake.
  • IO Resource: Flow
  • IO API Name: flows
  • Operations: Update

Prerequisites:

  • Create the table INTEGRATION_FLOWS in Snowflake before running the flow

DF08: Update Integrator IO Integration Exports/Imports to Snowflake Exports and Imports:

Technical Design:

This flow exports ‘exports/imports’ details from Snowflake and verifies if it is still available in IO. If export/import is deleted and not available in IO, it gets marked as inactive in Snowflake.

Introduction:

Sync integrator.io flow status to Snowflake: This integration flow syncs ‘exports/imports’ status from the integrator.io into Snowflake.

Integration Method:

  • Integration Flow Frequency: Batch.
  • Flow Direction: Export from Snowflake, Lookup in integrator.io and import into Snowflake.
  • IO Resource: Exports, Imports
  • IO API Name: exports, imports
  • Operations: Update

Prerequisites:

  • Create the table INTEGRATION_FLOW_EXPORTS_AND_IMPORTS in Snowflake before running the flow