> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wirekite.io/llms.txt
> Use this file to discover all available pages before exploring further.

# MySQL / MariaDB / SingleStore

> This guide explains how to configure MySQL, MariaDB, or SingleStore as a target database for Wirekite data loading and replication.

## Overview

Wirekite supports MySQL 5.x and above (including MariaDB and SingleStore) as a target database for:

* **Schema Loading** - Create target tables from Wirekite's intermediate schema format
* **Data Loading** - Bulk load extracted data using LOAD DATA INFILE
* **Change Loading (CDC)** - Apply ongoing changes using merge table approach

<Note>
  All references to MySQL in this guide also apply to MariaDB and SingleStore setups. SingleStore uses the MySQL wire protocol, so the configuration and parameters are identical. See the [SingleStore Notes](#singlestore-notes) section below for DSN and version differences.
</Note>

## Prerequisites

Before configuring MySQL as a Wirekite target, ensure the following requirements are met:

### Database Configuration

1. **Version**: MySQL 5.x or above (or MariaDB equivalent)
2. **User Permissions**: Create a Wirekite user with:
   * CREATE TABLE privilege
   * FILE privilege for LOAD DATA operations
   * INSERT, UPDATE, DELETE on target tables
   * Read/write access to Wirekite internal tables (`wirekite_progress` and `wirekite_action`)

### CLI Tools

Use the `mysql` command line tool or Wirekite cmdline tool to interact with the database.

<Tip>
  For cloud-managed MySQL (RDS, Cloud SQL), you'll need to use `databaseRemote=true` (the default) to use `LOAD DATA LOCAL INFILE`.
</Tip>

***

## Schema Loader

The Schema Loader reads Wirekite's intermediate schema format (`.skt` file) and generates MySQL-appropriate DDL statements for creating target tables.

<Note>
  If you need to add storage directives or other schema elements, use the CREATE TABLE output as a basis. Don't change column names, column order, or datatypes to incompatible ones.
</Note>

### Required Parameters

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file (`.skt`) generated by the Schema Extractor. Must be an absolute path.
</ResponseField>

<ResponseField name="createTableFile" type="string" required>
  Output file for CREATE TABLE statements. Includes both base tables and merge tables for CDC operations.
</ResponseField>

<ResponseField name="createConstraintFile" type="string" required>
  Output file for non-columnar constraints including indexes. Column-level constraints (NOT NULL) are in CREATE TABLE.
</ResponseField>

<ResponseField name="createForeignKeyFile" type="string" required>
  Output file for FOREIGN KEY definitions. Consider applying these after initial data load for better performance.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Schema Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="dropTableFile" type="string" default="none">
  Output file for DROP TABLE IF EXISTS statements. Set to "none" to skip generation.
</ResponseField>

<ResponseField name="createRecoveryTablesFile" type="string" default="none">
  Output file for recovery table creation SQL. Set to "none" to skip.
</ResponseField>

<ResponseField name="createMergeTables" type="boolean" default="true">
  When `true`, generates merge tables (`$wkm` suffix) for CDC operations. Set to `false` if only doing data loads.
</ResponseField>

<ResponseField name="mergeTablesOnly" type="boolean" default="false">
  When `true`, only creates merge tables, skipping base table creation.
</ResponseField>

***

## Data Loader

The Data Loader reads Wirekite's intermediate data format (`.dkt` files) and loads records into MySQL tables using LOAD DATA INFILE.

### Required Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the MySQL connection string.
</ResponseField>

**Connection string format:**

```
username:password@tcp(host:port)/database?multiStatements=true&loc=Local
```

**Example:**

```
wirekite:secretpass@tcp(mysql-target.example.com:3306)/myapp?multiStatements=true&loc=Local
```

<ResponseField name="inputDirectory" type="string" required>
  Directory containing data files (`.dkt`) to load.
</ResponseField>

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file used by Schema Loader. Required for table structure information.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Data Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="maxThreads" type="integer" default="5">
  Maximum number of parallel threads for loading tables. We recommend setting this to the number of CPUs on the host.
</ResponseField>

<ResponseField name="hexEncoding" type="boolean" default="false">
  Set to `true` if data was extracted using hex encoding instead of base64.
</ResponseField>

<ResponseField name="databaseRemote" type="boolean" default="true">
  When `true`, uses `LOAD DATA LOCAL INFILE`. When `false`, uses `LOAD DATA INFILE` (requires file access from server).
</ResponseField>

<ResponseField name="removeFiles" type="boolean" default="true">
  When `true`, removes data files from `inputDirectory` after loading.
</ResponseField>

***

## Change Loader

The Change Loader applies ongoing data changes (INSERT, UPDATE, DELETE) to MySQL tables using a merge approach with shadow tables.

<Note>
  The Change Loader uses merge tables to stage changes, then applies INSERT-SELECT, UPDATE, and DELETE operations to transfer changes to base tables.
</Note>

### Required Parameters

<ResponseField name="dsnFile" type="string" required>
  Path to a file containing the MySQL connection string.
</ResponseField>

**Connection string format:**

```
username:password@tcp(host:port)/database?multiStatements=true&loc=Local
```

<ResponseField name="inputDirectory" type="string" required>
  Directory containing change files (`.ckt`) from the Change Extractor.
</ResponseField>

<ResponseField name="workDirectory" type="string" required>
  Working directory for intermediate CSV files during merge operations.
</ResponseField>

<ResponseField name="schemaFile" type="string" required>
  Path to the Wirekite schema file for table structure information.
</ResponseField>

<ResponseField name="logFile" type="string" required>
  Absolute path to the log file for Change Loader operations.
</ResponseField>

### Optional Parameters

<ResponseField name="databaseRemote" type="boolean" default="true">
  When `true`, uses `LOAD DATA LOCAL INFILE` for loading changes to merge tables.
</ResponseField>

<ResponseField name="maxFilesPerBatch" type="integer" default="60">
  Maximum number of change files to process in a single batch before executing merge operations.
</ResponseField>

<ResponseField name="removeFiles" type="boolean" default="true">
  When `true`, removes change files from `inputDirectory` after fully processing.
</ResponseField>

<Warning>
  The Change Loader should not start until the Data Loader has successfully completed the initial full load.
</Warning>

***

## Orchestrator Configuration

When using the Wirekite Orchestrator, prefix target parameters with `target.schema.`, `target.data.`, or `target.change.`.

**Example orchestrator configuration for MySQL target:**

```
# Main configuration
source=oracle
target=mysql

# Schema loading
target.schema.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.schema.createTableFile=/opt/wirekite/output/schema/create_tables.sql
target.schema.createConstraintFile=/opt/wirekite/output/schema/constraints.sql
target.schema.createForeignKeyFile=/opt/wirekite/output/schema/foreign_keys.sql
target.schema.logFile=/var/log/wirekite/schema-loader.log

# Data loading
target.data.dsnFile=/opt/wirekite/config/mysql.dsn
target.data.inputDirectory=/opt/wirekite/output/data
target.data.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.data.logFile=/var/log/wirekite/data-loader.log
target.data.maxThreads=8
target.data.databaseRemote=true

# Change loading (CDC)
target.change.dsnFile=/opt/wirekite/config/mysql.dsn
target.change.inputDirectory=/opt/wirekite/output/changes
target.change.workDirectory=/opt/wirekite/work
target.change.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.change.logFile=/var/log/wirekite/change-loader.log
target.change.maxFilesPerBatch=30
target.change.databaseRemote=true
```

For complete Orchestrator documentation, see the [Execution Guide](/run/execution).

***

## SingleStore Notes

SingleStore (formerly MemSQL) uses the MySQL wire protocol and is fully compatible with the MySQL loaders described above. The following differences apply when targeting SingleStore.

### Version

SingleStore 7.x or above is required.

### Connection String

The SingleStore DSN must include `allowAllFiles=true` in addition to `multiStatements=true`. The `loc=Local` parameter used by MySQL is not needed.

**Connection string format:**

```
username:password@tcp(host:port)/database?allowAllFiles=true&multiStatements=true
```

**Example:**

```
wirekite:secretpass@tcp(singlestore.example.com:3306)/myapp?allowAllFiles=true&multiStatements=true
```

<Warning>
  The `allowAllFiles=true` parameter is required for LOAD DATA operations to work correctly with SingleStore. It is automatically added if missing.
</Warning>

### Orchestrator Configuration

When targeting SingleStore, use `target=singlestore` in the orchestrator configuration:

```
# Main configuration
source=mysql
target=singlestore

# Schema loading
target.schema.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.schema.createTableFile=/opt/wirekite/output/schema/create_tables.sql
target.schema.createConstraintFile=/opt/wirekite/output/schema/constraints.sql
target.schema.createForeignKeyFile=/opt/wirekite/output/schema/foreign_keys.sql
target.schema.logFile=/var/log/wirekite/schema-loader.log

# Data loading
target.data.dsnFile=/opt/wirekite/config/singlestore.dsn
target.data.inputDirectory=/opt/wirekite/output/data
target.data.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.data.logFile=/var/log/wirekite/data-loader.log
target.data.maxThreads=8
target.data.databaseRemote=true

# Change loading (CDC)
target.change.dsnFile=/opt/wirekite/config/singlestore.dsn
target.change.inputDirectory=/opt/wirekite/output/changes
target.change.workDirectory=/opt/wirekite/work
target.change.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.change.logFile=/var/log/wirekite/change-loader.log
target.change.maxFilesPerBatch=30
target.change.databaseRemote=true
```