Skip to main content

​
Overview

Wirekite supports PostgreSQL 10 and above (including YugaByte, TigerData, Google AlloyDB, and AWS Aurora PostgreSQL) as a target database for:
  • Schema Loading - Create target tables from Wirekite’s intermediate schema format
  • Data Loading - Bulk load extracted data using COPY command
  • Change Loading (CDC) - Apply ongoing changes using merge table approach
PostgreSQL loaders use the high-performance COPY command for bulk loading. The Change Loader uses shadow tables with _wkm suffix for atomic change application.

​
Prerequisites

Before configuring PostgreSQL as a Wirekite target, ensure the following requirements are met:

​
Database Configuration

  1. Version: PostgreSQL 10 or above
  2. User Permissions: Create a Wirekite user with:
    • CREATE TABLE privilege
    • FILE access for COPY operations
    • INSERT, UPDATE, DELETE on target tables
    • Read/write access to Wirekite internal tables (wirekite_progress and wirekite_action)

​
CLI Tools

Use the psql command line tool or Wirekite cmdline tool to interact with PostgreSQL.
For cloud-managed PostgreSQL (AlloyDB, Aurora, Cloud SQL), you’ll need to use databaseRemote=true (the default) since server-side file access isn’t available.

​
Schema Loader

The Schema Loader reads Wirekite’s intermediate schema format (.skt file) and generates PostgreSQL-appropriate DDL statements for creating target tables.
If you need to add storage directives or other schema elements, use the CREATE TABLE output as a basis. Don’t change column names, column order, or datatypes to incompatible ones.

​
Required Parameters

​
schemaFile
string
required
Path to the Wirekite schema file (.skt) generated by the Schema Extractor. Must be an absolute path.
​
createTableFile
string
required
Output file for CREATE TABLE statements. Includes both base tables and merge tables for CDC operations.
​
createConstraintFile
string
required
Output file for non-columnar constraints including indexes. Column-level constraints (NOT NULL) are in CREATE TABLE.
​
createForeignKeyFile
string
required
Output file for FOREIGN KEY definitions. Consider applying these after initial data load for better performance.
​
logFile
string
required
Absolute path to the log file for Schema Loader operations.

​
Optional Parameters

​
dropTableFile
string
default:"none"
Output file for DROP TABLE IF EXISTS statements. Set to “none” to skip generation.
​
createRecoveryTablesFile
string
default:"none"
Output file for recovery table creation SQL. Set to “none” to skip.
​
createMergeTables
boolean
default:"true"
When true, generates merge tables for CDC operations. Set to false if only doing data loads.

​
Extended Schema Options

When migrating from Oracle, PostgreSQL can generate additional DDL for non-table objects. Set each parameter to a file path to enable, or leave as "none" to skip.
​
objectsFile
string
default:"none"
Path to the source XKT file containing extended schema objects (views, procedures, etc.).
​
createIndexFile
string
default:"none"
Output file for CREATE INDEX statements.
​
createSequenceFile
string
default:"none"
Output file for CREATE SEQUENCE statements.
​
createViewsFile
string
default:"none"
Output file for CREATE VIEW statements.
​
createMaterializedViewsFile
string
default:"none"
Output file for CREATE MATERIALIZED VIEW statements.
​
createSynonymsFile
string
default:"none"
Output file for CREATE SYNONYM statements.
​
createGrantsFile
string
default:"none"
Output file for GRANT statements.
​
createRolesUsersFile
string
default:"none"
Output file for role and user creation statements.
​
createSchemasFile
string
default:"none"
Output file for CREATE SCHEMA statements.
​
createTriggersFile
string
default:"none"
Output file for CREATE TRIGGER statements.
​
createProceduresFile
string
default:"none"
Output file for CREATE PROCEDURE statements.
​
createFunctionsFile
string
default:"none"
Output file for CREATE FUNCTION statements.
​
createPackagesFile
string
default:"none"
Output file for CREATE PACKAGE statements.

​
Data Loader

The Data Loader reads Wirekite’s intermediate data format (.dkt files) and loads records into PostgreSQL tables using COPY commands.

​
Required Parameters

​
dsnFile
string
required
Path to a file containing the PostgreSQL connection string.
Connection string format: 
postgres://username:password@host:port/database
Example: 
postgres://wirekite:secretpass@postgres-target.example.com:5432/myapp
​
inputDirectory
string
required
Directory containing data files (.dkt) to load.
​
schemaFile
string
required
Path to the Wirekite schema file used by Schema Loader. Required for table structure information.
​
logFile
string
required
Absolute path to the log file for Data Loader operations.

​
Optional Parameters

​
maxThreads
integer
default:"5"
Maximum number of parallel threads for loading tables. We recommend setting this to the number of CPUs on the host.
​
hexEncoding
boolean
default:"false"
Set to true if data was extracted using hex encoding instead of base64.
​
databaseRemote
boolean
default:"true"
When true, uses COPY FROM STDIN. When false, uses local file I/O methods.
​
removeFiles
boolean
default:"true"
When true, removes data files from inputDirectory after loading. Should typically be true to save disk space.

​
Change Loader

The Change Loader applies ongoing data changes (INSERT, UPDATE, DELETE) to PostgreSQL tables using a merge approach with shadow tables.
The Change Loader uses merge tables to stage changes before applying them atomically to base tables.

​
Required Parameters

​
dsnFile
string
required
Path to a file containing the PostgreSQL connection string.
Connection string format: 
postgres://username:password@host:port/database
​
inputDirectory
string
required
Directory containing change files (.ckt) from the Change Extractor.
​
workDirectory
string
required
Working directory for intermediate files during merge operations.
​
schemaFile
string
required
Path to the Wirekite schema file for table structure information.
​
logFile
string
required
Absolute path to the log file for Change Loader operations.

​
Optional Parameters

​
databaseRemote
boolean
default:"true"
When true, indicates remote database connection mode.
​
maxFilesPerBatch
integer
default:"30"
Maximum number of change files to process in a single batch.
​
removeFiles
boolean
default:"true"
When true, removes change files from inputDirectory after fully processing. Should typically be true to save disk space.
The Change Loader should not start until the Data Loader has successfully completed the initial full load.

​
Orchestrator Configuration

When using the Wirekite Orchestrator, prefix target parameters with target.schema., target.data., or target.change.. Example orchestrator configuration for PostgreSQL target: 
# Main configuration
source=oracle
target=postgres

# Schema loading
target.schema.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.schema.createTableFile=/opt/wirekite/output/schema/create_tables.sql
target.schema.createConstraintFile=/opt/wirekite/output/schema/constraints.sql
target.schema.createForeignKeyFile=/opt/wirekite/output/schema/foreign_keys.sql
target.schema.logFile=/var/log/wirekite/schema-loader.log

# Data loading
target.data.dsnFile=/opt/wirekite/config/postgres.dsn
target.data.inputDirectory=/opt/wirekite/output/data
target.data.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.data.logFile=/var/log/wirekite/data-loader.log
target.data.maxThreads=8
target.data.databaseRemote=true

# Change loading (CDC)
target.change.dsnFile=/opt/wirekite/config/postgres.dsn
target.change.inputDirectory=/opt/wirekite/output/changes
target.change.workDirectory=/opt/wirekite/work
target.change.schemaFile=/opt/wirekite/output/schema/wirekite_schema.skt
target.change.logFile=/var/log/wirekite/change-loader.log
target.change.maxFilesPerBatch=30
target.change.databaseRemote=true
For complete Orchestrator documentation, see the Execution Guide.