> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wirekite.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Google Cloud Spanner

> This guide explains how to configure Google Cloud Spanner as a target database for Wirekite data loading and replication.

## Overview

Wirekite supports Google Cloud Spanner as a target database for:

* **Schema Loading** - Create target tables from Wirekite's intermediate schema format
* **Data Loading** - Bulk load extracted data using Spanner mutations
* **Change Loading (CDC)** - Apply ongoing changes using shadow table merge approach

<Note>
  Spanner loaders respect Spanner's mutation limits (80,000 mutations per commit). The Change Loader uses shadow tables with `_wkm` suffix for atomic change application.
</Note>

## Prerequisites

Before configuring Spanner as a Wirekite target, ensure the following requirements are met:

### Google Cloud Configuration

1. **Project Setup**: Have a Google Cloud project with Spanner API enabled
2. **Instance & Database**: Create a Spanner instance and database
3. **Authentication**: Configure Application Default Credentials or service account
4. **IAM Permissions**: Ensure the service account has:
   * `spanner.databases.read`
   * `spanner.databases.write`
   * `spanner.sessions.create`
5. **Metadata Tables**: Wirekite automatically creates `wirekite_progress` and `wirekite_action` tables in the target database for tracking migration state and operational control. See [Operations](/run/operations) for details.

### Spanner Limitations

<Warning>
  Spanner has specific limitations to be aware of:

  * Maximum STRING/BYTES length: 2,621,440 bytes (2.5MB)
  * Maximum mutations per commit: 80,000 (counting columns)
  * Change streams have maximum 7-day retention
</Warning>

<Tip>
  Use Application Default Credentials for simplest authentication: run `gcloud auth application-default login` on the loader host.
</Tip>

***

## Schema Loader

The Schema Loader reads Wirekite's intermediate schema format (`.skt` file) and generates Spanner-appropriate DDL statements for creating target tables.

<Note>
  The Schema Loader can also generate Change Stream DDL for Spanner-to-Spanner replication scenarios.
</Note>

### Required Parameters

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file (`.skt`) generated by the Schema Extractor. Must be an absolute path.
</ResponseField>

<ResponseField name="createTableFile" type="string" required>
  Output file for CREATE TABLE statements. Includes both base tables and merge tables for CDC operations.
</ResponseField>

<ResponseField name="createConstraintFile" type="string" required>
  Output file for constraint definitions (indexes, unique constraints).
</ResponseField>

<ResponseField name="createForeignKeyFile" type="string" required>
  Output file for FOREIGN KEY constraints (interleaved tables in Spanner).
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Schema Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="dropTableFile" type="string" default="none">
  Output file for DROP TABLE statements. Set to "none" to skip generation.
</ResponseField>

<ResponseField name="createRecoveryTablesFile" type="string" default="none">
  Output file for recovery table creation DDL. Set to "none" to skip.
</ResponseField>

<ResponseField name="createChangeStreamFile" type="string" default="none">
  Output file for CREATE CHANGE STREAM statement. Used when Spanner is also a source.
</ResponseField>

<ResponseField name="dropChangeStreamFile" type="string" default="none">
  Output file for DROP CHANGE STREAM statement. Set to "none" to skip.
</ResponseField>

<ResponseField name="changeStreamRetention" type="string" default="7d">
  Change stream retention period. Maximum is "7d" (7 days) for Spanner.
</ResponseField>

<ResponseField name="createMergeTables" type="boolean" default="true">
  When `true`, generates merge tables (`_wkm` suffix) for CDC operations. Set to `false` if only doing data loads.
</ResponseField>

***

## Data Loader

The Data Loader reads Wirekite's intermediate data format (`.dkt` files) and loads records into Spanner tables using batched mutations.

### Required Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the Spanner connection string.
</ResponseField>

**Connection string format:**

```
projects/PROJECT_ID/instances/INSTANCE_ID/databases/DATABASE_ID
```

**Example:**

```
projects/my-project/instances/my-instance/databases/my-database
```

<ResponseField name="inputDirectory" type="string" required>
  Directory containing data files (`.dkt`) to load.
</ResponseField>

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file used by Schema Loader. Required for table structure information.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Data Loader operations.
</ResponseField>

<ResponseField name="gcpCredfile" type="string" required>
  Absolute path to a GCP service-account JSON credentials file. Used to authenticate the Spanner client. If the loader host already has Application Default Credentials configured (e.g. via `gcloud auth application-default login` or a workload-identity-bound VM), point this at the ADC file path — the loader will not start without an explicit value.
</ResponseField>

### Optional Parameters

<ResponseField name="maxThreads" type="integer" default="5">
  Maximum number of parallel threads for loading tables.
</ResponseField>

<ResponseField name="hexEncoding" type="boolean" default="false">
  Set to `true` if data was extracted using hex encoding instead of base64.
</ResponseField>

<Tip>
  The Data Loader automatically batches mutations to stay within Spanner's 80,000 mutation limit per commit.
</Tip>

***

## Change Loader

The Change Loader applies ongoing data changes (INSERT, UPDATE, DELETE) to Spanner tables using a merge approach with shadow tables.

### Required Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the Spanner connection string.
</ResponseField>

**Connection string format:**

```
projects/PROJECT_ID/instances/INSTANCE_ID/databases/DATABASE_ID
```

<ResponseField name="inputDirectory" type="string" required>
  Directory containing change files (`.ckt`) from the Change Extractor.
</ResponseField>

<ResponseField name="workDirectory" type="string" required>
  Working directory for temporary files during merge operations. Must be writable.
</ResponseField>

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file for table structure information.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Change Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="loadLocal" type="boolean" default="true">
  When `true`, loads changes locally.
</ResponseField>

<ResponseField name="maxFilesPerBatch" type="integer" default="30">
  Maximum number of change files to process in a single batch.
</ResponseField>

<ResponseField name="doMerge" type="boolean" default="true">
  When `true`, uses shadow table merge approach. When `false`, uses direct mutations.
</ResponseField>

<Warning>
  The Change Loader should not start until the Data Loader has successfully completed the initial full load.
</Warning>

***

## Orchestrator Configuration

When using the Wirekite Orchestrator, prefix target parameters with `target.schema.`, `target.data.`, or `target.change.`.

**Example orchestrator configuration for Spanner target:**

```
# Main configuration
source=postgres
target=spanner

# Schema loading
target.schema.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.schema.createTableFile=/opt/wirekite/output/schema/create_tables.sql
target.schema.createConstraintFile=/opt/wirekite/output/schema/constraints.sql
target.schema.createForeignKeyFile=/opt/wirekite/output/schema/foreign_keys.sql
target.schema.logFile=/var/log/wirekite/schema-loader.log

# Data loading
target.data.dsnFile=/opt/wirekite/config/spanner.dsn
target.data.inputDirectory=/opt/wirekite/output/data
target.data.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.data.logFile=/var/log/wirekite/data-loader.log
target.data.gcpCredfile=/opt/wirekite/config/gcp-sa.json
target.data.maxThreads=8

# Change loading (CDC)
target.change.dsnFile=/opt/wirekite/config/spanner.dsn
target.change.inputDirectory=/opt/wirekite/output/changes
target.change.workDirectory=/opt/wirekite/work
target.change.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.change.logFile=/var/log/wirekite/change-loader.log
target.change.maxFilesPerBatch=30
```

For complete Orchestrator documentation, see the [Execution Guide](/run/execution).