> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wirekite.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Snowflake

> This guide explains how to configure Snowflake as a target data warehouse for Wirekite data loading and replication.

## Overview

Wirekite supports Snowflake as a target data warehouse for:

* **Schema Loading** - Create target tables from Wirekite's intermediate schema format
* **Data Loading** - Bulk load extracted data via Snowflake internal stage
* **Change Loading (CDC)** - Apply ongoing changes using MERGE operations

<Note>
  Snowflake loaders stage data through Snowflake's internal stage using PUT commands, then use COPY INTO for high-performance bulk loading.
</Note>

## Prerequisites

Before configuring Snowflake as a Wirekite target, ensure the following requirements are met:

### Snowflake Configuration

1. **User Setup**: Create a properly configured Snowflake user with appropriate privileges
2. **Warehouse**: Ensure a virtual warehouse is available for loading operations
3. **Database & Schema**: Create the target database and schema
4. **CLI Tools**: Either install the [SnowSQL command line tool](https://docs.snowflake.com/en/user-guide/snowsql-install-config) or use the Wirekite cmdline tool

### Internal Tables

<Warning>
  Ensure the Wirekite target metadata tables (`wirekite_progress` and `wirekite_action`) exist in your Snowflake database and are read-write accessible. Use the Wirekite cmdline tool to verify connectivity.
</Warning>

<Tip>
  For connection string format details, see the [Snowflake connection documentation](https://docs.snowflake.com/en/user-guide/gen-conn-config).
</Tip>

***

## Schema Loader

The Schema Loader reads Wirekite's intermediate schema format (`.skt` file) and generates Snowflake-appropriate DDL statements for creating target tables.

<Note>
  The Schema Loader generates both base tables and merge tables (with `$wkm` suffix) for CDC operations.
</Note>

### Required Parameters

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file (`.skt`) generated by the Schema Extractor. Must be an absolute path.
</ResponseField>

<ResponseField name="createTableFile" type="string" required>
  Output file for CREATE TABLE statements. Includes both base tables and merge tables for CDC operations.
</ResponseField>

<ResponseField name="createConstraintFile" type="string" required>
  Output file for CHECK constraints. Snowflake doesn't enforce many constraints, so you may choose not to load this file.
</ResponseField>

<ResponseField name="createForeignKeyFile" type="string" required>
  Output file for FOREIGN KEY constraints. You may elect not to load foreign keys if you don't need them.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Schema Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="dropTableFile" type="string" default="none">
  Output file for DROP TABLE IF EXISTS statements. Set to "none" to skip generation.
</ResponseField>

<ResponseField name="createRecoveryTablesFile" type="string" default="none">
  Output file for recovery table creation SQL. Set to "none" to skip.
</ResponseField>

<ResponseField name="createMergeTables" type="boolean" default="true">
  When `true`, generates merge tables (`$wkm` suffix) for CDC operations. Set to `false` if only doing data loads without change capture.
</ResponseField>

***

## Data Mover

The Data Mover uploads extracted data files to Snowflake's internal stage using PUT commands for subsequent loading.

### Required Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the Snowflake connection string.
</ResponseField>

**Connection string format (Golang connector):**

```
<snowflake-user>:<snowflake-pwd>@<snowflake-orgname>-<snowflake-acct>/<database>/<schema>?warehouse=<warehouse-name>
```

**Example:**

```
wirekite:secretpass@myorg-myaccount/MYDB/PUBLIC?warehouse=COMPUTE_WH
```

<ResponseField name="dataDirectory" type="string" required>
  Directory where the Data Extractor wrote its files. These will be copied to the Snowflake staging area.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Data Mover operations.
</ResponseField>

### Optional Parameters

<ResponseField name="maxThreads" type="integer" default="10">
  Maximum number of parallel threads for uploading to Snowflake stage.
</ResponseField>

<ResponseField name="gzipFiles" type="boolean" default="false">
  When `true`, compresses files before uploading. Changes extension to `.dgz`.
</ResponseField>

<ResponseField name="removeFiles" type="boolean" default="false">
  When `true`, deletes local files after successful PUT to stage. Should typically be `true` in production to save disk space.
</ResponseField>

***

## Data Loader

The Data Loader reads data files from Snowflake's internal stage and loads them into target tables using COPY INTO commands.

### Required Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the Snowflake connection string.
</ResponseField>

**Connection string format (Golang connector):**

```
<snowflake-user>:<snowflake-pwd>@<snowflake-orgname>-<snowflake-acct>/<database>/<schema>?warehouse=<warehouse-name>
```

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file used by Schema Loader. Required for table structure information.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Data Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="maxThreads" type="integer" default="5">
  Maximum number of parallel COPY threads. We recommend setting this to the number of CPUs on the host.
</ResponseField>

<ResponseField name="hexEncoding" type="boolean" default="false">
  Set to `true` if data was extracted using hex encoding instead of base64.
</ResponseField>

<ResponseField name="dataFileExtension" type="string" default="dkt">
  File extension for data files to process (e.g., "dkt", "dgz").
</ResponseField>

<Tip>
  For best performance, set `maxThreads` equal to the number of CPU cores available on the loader host.
</Tip>

***

## Change Loader

The Change Loader applies ongoing data changes (INSERT, UPDATE, DELETE) to Snowflake tables using MERGE operations with shadow tables.

<Note>
  The Change Loader uses a merging approach that stages intermediate data to Snowflake's internal stage before executing MERGE statements.
</Note>

### Required Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the Snowflake connection string.
</ResponseField>

**Connection string format (Golang connector):**

```
<snowflake-user>:<snowflake-pwd>@<snowflake-orgname>-<snowflake-acct>/<database>/<schema>?warehouse=<warehouse-name>
```

<ResponseField name="inputDirectory" type="string" required>
  Directory where the Change Extractor wrote its files. These will be sourced for changes.
</ResponseField>

<ResponseField name="workDirectory" type="string" required>
  Working directory for intermediate files that are uploaded to Snowflake stage during merge operations.
</ResponseField>

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file for table structure information.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Change Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="maxFilesPerBatch" type="integer" default="60">
  Maximum number of change files to process in a single batch before executing MERGE operations.
</ResponseField>

<ResponseField name="maxMergeThreads" type="integer">
  Number of parallel threads for applying merge operations within each batch. Defaults to 2x the number of CPU cores on the host.
</ResponseField>

<ResponseField name="hexEncoding" type="boolean" default="false">
  Set to `true` if change data was extracted using hex encoding.
</ResponseField>

<ResponseField name="removeFiles" type="boolean" default="true">
  When `true`, removes change files from `inputDirectory` after fully processing. Should typically be `true` in production to save disk space.
</ResponseField>

<Warning>
  The Change Loader should not start until the Data Loader has successfully completed the initial full load.
</Warning>

***

## Orchestrator Configuration

When using the Wirekite Orchestrator, prefix parameters with `mover.`, `target.schema.`, `target.data.`, or `target.change.`.

**Example orchestrator configuration for Snowflake target:**

```
# Main configuration
source=postgres
target=snowflake

# Data mover
mover.dsnFile=/opt/wirekite/config/snowflake.dsn
mover.dataDirectory=/opt/wirekite/output/data
mover.logFile=/var/log/wirekite/data-mover.log
mover.maxThreads=10
mover.removeFiles=true

# Schema loading
target.schema.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.schema.createTableFile=/opt/wirekite/output/schema/create_tables.sql
target.schema.createConstraintFile=/opt/wirekite/output/schema/constraints.sql
target.schema.createForeignKeyFile=/opt/wirekite/output/schema/foreign_keys.sql
target.schema.logFile=/var/log/wirekite/schema-loader.log

# Data loading
target.data.dsnFile=/opt/wirekite/config/snowflake.dsn
target.data.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.data.logFile=/var/log/wirekite/data-loader.log
target.data.maxThreads=8

# Change loading (CDC)
target.change.dsnFile=/opt/wirekite/config/snowflake.dsn
target.change.inputDirectory=/opt/wirekite/output/changes
target.change.workDirectory=/opt/wirekite/work
target.change.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.change.logFile=/var/log/wirekite/change-loader.log
target.change.maxFilesPerBatch=30
```

For complete Orchestrator documentation, see the [Execution Guide](/run/execution).