> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wirekite.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Snowflake Data Mover

> Transfers extracted data files directly to Snowflake internal stages using PUT commands.

The Snowflake Data Mover transfers files extracted by Wirekite Extractors directly to Snowflake's internal stage using native PUT commands. This is the recommended mover for Snowflake targets as it leverages Snowflake's optimized file transfer mechanism.

The mover watches a local directory for extracted data files, uploads them to the user's Snowflake stage (`@~`), and optionally compresses and removes local files after successful upload.

## When to Use

Use the Snowflake Data Mover when:

* Snowflake is your target database
* You want to use Snowflake's native PUT command for optimal performance
* You prefer staging data in Snowflake's internal stage rather than external cloud storage

## Features

* **Native Snowflake PUT** - Uses Snowflake's optimized file transfer protocol
* **Multi-threaded uploads** - Parallel uploads for better throughput
* **Optional gzip compression** - Compress files before upload
* **Crash recovery** - Safe restarts without re-uploading completed files
* **Automatic cleanup** - Removes local files after successful upload
* **Coordinated completion** - Waits for `DATA.DONE` signal before finalizing

## Configuration

The Snowflake Data Mover uses a configuration file passed as a command-line argument:

```bash theme={null}
snowflake_data_mover <config_file>
```

### Configuration Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the Snowflake connection string (DSN). The file should contain a single line with the full Snowflake connection URL.
</ResponseField>

<ResponseField name="dataDirectory" type="string" required>
  The full path on the local machine where the Wirekite Data Extractor writes its files.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  The path to a file where the Snowflake Data Mover will write its logging output.
</ResponseField>

<ResponseField name="maxThreads" type="integer">
  Maximum number of parallel threads for uploading files. Default is `10`.
</ResponseField>

<ResponseField name="gzipFiles" type="boolean">
  Whether to gzip files before uploading. When enabled, files are compressed locally and uploaded with `SOURCE_COMPRESSION=GZIP`. Default is `false`.
</ResponseField>

<ResponseField name="removeFiles" type="boolean">
  Whether to remove local files after successful upload. Default is `false`.
</ResponseField>

### Example Configuration

```
dsnFile = /etc/wirekite/snowflake_dsn.txt
dataDirectory = /data/wirekite/extract
logFile = /var/log/wirekite/snowflake_mover.log
maxThreads = 15
gzipFiles = true
removeFiles = true
```

### Snowflake DSN File Format

The DSN file should contain a single line with your Snowflake connection string:

```
user:password@account/database/schema?warehouse=COMPUTE_WH
```

Or with additional parameters:

```
user:password@account.region.cloud/database/schema?warehouse=COMPUTE_WH&role=DATA_LOADER
```

## Orchestrator Integration

When using the Snowflake Data Mover with the Wirekite Orchestrator, add the mover configuration to your orchestrator config:

```
moverType = snowflake
moverConfig = /path/to/snowflake_mover.cfg
```

## How It Works

1. The mover connects to Snowflake using the provided DSN
2. Monitors `dataDirectory` for files with the specified extension
3. For each file found:
   * Optionally compresses the file with gzip
   * Executes a PUT command to upload to `@~/filename`
   * Verifies the upload status is `UPLOADED`
   * Removes the local file (if configured)
4. When `DATA.DONE` appears in the directory, uploads it and exits gracefully

### PUT Command Details

The mover uses different PUT commands based on compression settings:

**Without compression:**

```sql theme={null}
PUT file:///path/to/file.dkt @~/file.dkt SOURCE_COMPRESSION=NONE AUTO_COMPRESS=FALSE;
```

**With compression:**

```sql theme={null}
PUT file:///path/to/file.dgz @~/file.dgz SOURCE_COMPRESSION=GZIP;
```

## Notes

* Files are uploaded to the user's default stage (`@~`)
* The Snowflake user must have permissions to PUT files to their stage
* For large migrations, increase `maxThreads` to improve throughput
* Enable `gzipFiles` to reduce network transfer time
* The Snowflake Loader should be configured to read from the same stage
* This mover requires direct network connectivity to Snowflake from the extraction host