Snowflake

You can configure Snowflake only as a target.

To configure the connection details for Snowflake in your Micro-Integration, see Snowflake Connection Parameters.

You must also define at least one Micro-Integration Flow that specifies:

For message headers, see Snowflake Message Headers.

Snowflake Prerequisites

Before you configure this Micro-Integration, complete the following setup steps in Snowflake:

  1. Create the target database and schema, if they don't already exist:

    CREATE DATABASE IF NOT EXISTS <database>;
    CREATE SCHEMA IF NOT EXISTS <database>.<schema>;
  2. Create the target table, or let the Micro-Integration create it for you. The Micro-Integration always writes to a table with the following fixed schema:

    CREATE TABLE <database>.<schema>.<table> (
        RECORD_CONTENT  VARIANT,
        RECORD_METADATA VARIANT
    );

    RECORD_CONTENT contains the output of the mapping stage. Structured payloads (maps, collections, arrays) are serialized to JSON; string and binary payloads are written as-is. RECORD_METADATA contains any message headers that are explicitly mapped. Headers are not automatically populated.

    Instead of creating the table manually, you can set provisioning-mode.tables to CREATE on the workflow's producer binding so the Micro-Integration creates the table automatically using the schema above. The default, NONE, requires the table to already exist.

  3. Create a dedicated Snowflake user for the Micro-Integration, rather than reusing a personal or shared account. A dedicated user isolates the Micro-Integration's access so you can audit, rotate, or revoke its credentials independently:

    CREATE USER <mi_user>
      DEFAULT_ROLE = <mi_role>
      MUST_CHANGE_PASSWORD = FALSE;
  4. Create a role for the Micro-Integration and grant it the privileges it needs to write to the target table. Grant CREATE TABLE only if you're using provisioning-mode.tables: CREATE; otherwise, pre-create the table and grant INSERT on it directly:

    CREATE ROLE <mi_role>;
    GRANT USAGE ON WAREHOUSE <warehouse> TO ROLE <mi_role>;
    GRANT USAGE ON DATABASE <database> TO ROLE <mi_role>;
    GRANT USAGE ON SCHEMA <database>.<schema> TO ROLE <mi_role>;
    GRANT INSERT ON TABLE <database>.<schema>.<table> TO ROLE <mi_role>;
    GRANT CREATE TABLE ON SCHEMA <database>.<schema> TO ROLE <mi_role>; -- only if using provisioning-mode.tables: CREATE
    
    GRANT ROLE <mi_role> TO USER <mi_user>;
    ALTER USER <mi_user> SET DEFAULT_WAREHOUSE = <warehouse>;

    The Micro-Integration has no warehouse property, so it relies on the user or role having a default warehouse assigned in Snowflake, as shown above. It only ever writes to the target table and never queries it, so the privileges shown above are all it needs.

    Use a role scoped to only these privileges in production. ACCOUNTADMIN and other broad system roles are useful for initial testing, but shouldn't be used for a running Micro-Integration.

  5. Set up key-pair authentication. The Micro-Integration requires Snowflake key-pair authentication; username and password authentication is not supported. Generate an RSA key pair using OpenSSL:

    openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt

    To encrypt the private key with a passphrase instead, omit -nocrypt:

    openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -v2 aes-256-cbc -out rsa_key.p8

    Then generate the matching public key:

    openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub

    The private key doesn't have to be in PKCS#8 format. The Micro-Integration also accepts traditional PEM-encoded RSA keys, whether encrypted or unencrypted. PKCS#8, produced by the commands above, is Snowflake's recommended format, not a requirement.

    Assign the public key to the Snowflake user, using only the key body (not the -----BEGIN/END PUBLIC KEY----- lines) as a single unbroken string:

    ALTER USER <mi_user> SET RSA_PUBLIC_KEY='<contents of rsa_key.pub>';

    Provide the contents of rsa_key.p8 and, if you encrypted it, its passphrase, as the Private Key and Private Key Password connection parameters.

  6. Retrieve the values you'll need to configure the connection:

    • Your account URL, in the format <locator>.<region>.snowflakecomputing.com:443. You can find your account locator and region by running SELECT CURRENT_ACCOUNT(), CURRENT_REGION();, or from the account selector in the Snowflake web interface.

    • The database, schema, and table names you created in step 1 (or that the Micro-Integration will auto-create, if you're using provisioning-mode.tables: CREATE).

  7. Verify your setup by connecting to Snowflake as <mi_user> (for example, using SnowSQL or the Snowflake web interface) and confirming you can reach the target objects before you continue to configuration:

    USE ROLE <mi_role>;
    USE WAREHOUSE <warehouse>;
    INSERT INTO <database>.<schema>.<table> (RECORD_CONTENT, RECORD_METADATA) VALUES (PARSE_JSON('{}'), PARSE_JSON('{}'));
    SELECT * FROM <database>.<schema>.<table>;

    If this succeeds, remove the test row before configuring the Micro-Integration: TRUNCATE TABLE <database>.<schema>.<table>;

Snowflake Connection Parameters

The following table describes the connection parameters for Snowflake.

Field Description

Snowflake URL

The URL of the Snowflake instance. For example, the format of the URL can be <LOCATOR>.<REGION>.snowflakecomputing.com:443. For more information about the URL and account information, see the Snowflake documentation.

Snowflake Username

The username to log in to Snowflake.

Role

The type of role in Snowflake. For example, ACCOUNTADMIN, SECURITYADMIN, USERADMIN, SYSADMIN, or PUBLIC. For more information about the Snowflake roles, see the Snowflake documentation.

Private Key

The path to the private key file. For more information about the private key path, see the Snowflake documentation.

Private Key Password

The password for the private key, if applicable. For more information about the private key password, see the Snowflake documentation.

Micro-Integration Flow Parameters

You must configure the endpoint parameters for each Flow. Each Flow can have different settings, but they all share the connection details of the parent Micro-Integration.

Snowflake Target Parameters

The following table describes the parameters for configuring Snowflake as a target.

Field Description
Database

The uppercase Snowflake database name.

Alternatively, you can set the Smart Topic Destination on the Mapping step to a fully-qualified table name (database.schema.table), which will override this destination.

Schema

The uppercase Snowflake schema name.

Alternatively, you can set the Smart Topic Destination on the Mapping step to a fully-qualified table name (database.schema.table), which will override this destination.

Table

The uppercase Snowflake table name.

Alternatively, you can set the Smart Topic Destination on the Mapping step to a fully-qualified table name (database.schema.table), which will override this destination.

Provisioning Mode

Specifies whether and how the Micro-Integration provisions the specified target table:

  • NONE—the Micro-Integration does not attempt to create the table.

  • CREATE—the table is created if it does not exist. Auto-provisioned tables use the schema: RECORD_CONTENT VARIANT (output of the mapping stage; structured payloads serialized to JSON, strings and binary written as-is) and RECORD_METADATA VARIANT (explicitly mapped message headers).

Acknowledgment Mode

Controls when the Micro-Integration sends acknowledgments to the event broker service:

  • ON_COMMIT—the Micro-Integration waits until a row has been successfully committed to the target table before sending a message acknowledgment.

  • ON_INSERT—the acknowledgment is sent as soon as the message has been written to the Snowflake channel or stage.

Troubleshooting

Key-Pair Authentication Failures
If the private key is encrypted, confirm the passphrase you supplied matches the one used to encrypt it.
Confirm the public key currently set on the Snowflake user (DESCRIBE USER <mi_user>, checking RSA_PUBLIC_KEY_FP) matches the private key file the Micro-Integration is using. If you rotated the key pair, update whichever side is out of date.
Streaming Insert Failures
If the error indicates the target table doesn't exist, confirm the table has been created, or that provisioning-mode.tables is set to CREATE so the Micro-Integration creates it automatically.
Confirm the database, schema, and table names are correct and use the expected case. Unquoted Snowflake identifiers are stored in uppercase by default.
Confirm the role assigned to the Micro-Integration's Snowflake user has the required privileges on the target objects. See Insufficient Privileges below.
Insufficient Privileges
Confirm the role has been granted USAGE on the warehouse, database, and schema, and either INSERT on the target table or CREATE TABLE on the schema, depending on your provisioning mode.
Confirm the role has been granted to the user, and that the user's default role or session role is set correctly.
Broad system roles such as ACCOUNTADMIN are useful for initial testing, but a dedicated, least-privilege role is recommended once you deploy to production.
Messages Not Appearing in the Target Table
Confirm the destination configured for the workflow matches the database, schema, and table you expect. A fully-qualified destination overrides any separately configured database, schema, or table values.
Query the table directly (SELECT * FROM <database>.<schema>.<table>;) rather than relying on a cached view, since writes are committed as regular table inserts and are visible as soon as the Micro-Integration's acknowledgment mode confirms the write.