> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wirekite.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting

> Diagnose and resolve common Wirekite migration issues.

## Exit Codes

| Code | Meaning                                                                                           | Action                                    |
| ---- | ------------------------------------------------------------------------------------------------- | ----------------------------------------- |
| 0    | Success                                                                                           | None                                      |
| 1    | Operational error (connection failure, config error, I/O error, license validation failure, etc.) | Check logs for the specific error message |
| 2    | Usage cap exceeded                                                                                | Check usage limits or upgrade license     |

## Log Files

Each component writes to a log file specified by the `logFile` parameter in the orchestrator configuration. For migrations created through the [Web Interface](/run/execution/ux), logs are stored under the migration directory.

**Per-migration log structure:**

```
/opt/wirekite/instance/<migration-name>/
├── schema/logs/
├── data/logs/
├── change/logs/
└── validate/logs/
```

**UX Server logs:**

```
/opt/wirekite/ux/logs/ux-server.log
/opt/wirekite/ux/logs/ux-server-error.log
```

**Usage tracking:**

```
/opt/wirekite/instance/usage.log
```

<Tip>
  You can download all logs for a migration as a ZIP file through the Web Interface by opening the migration and clicking **Download Logs**.
</Tip>

## License Issues

### License Check Failed (Exit Code 1)

The orchestrator validates the license at startup. If validation fails, it prints an error and exits with code 1. Note that exit code 1 is also used for other operational errors (connection failures, configuration errors, file I/O errors, etc.) -- check the log message for the specific cause.

| Error Message                          | Cause                                                 | Fix                                                          |
| -------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------ |
| `failed to read license file`          | License file missing or unreadable                    | Verify files exist at `/opt/wirekite/license/` with mode 600 |
| `license file has been tampered with`  | HMAC signature mismatch                               | Reinstall license with `install-license.sh`                  |
| `License has expired`                  | Current date is past the expiry date                  | Contact your license provider for renewal                    |
| `License issued date is in the future` | System clock is set incorrectly                       | Fix system clock with `timedatectl` or NTP                   |
| `Host ID mismatch`                     | License is bound to a different machine's MAC address | Contact provider for a license matching this host            |

**Diagnose with:**

```bash theme={null}
/opt/wirekite/license/license -cmd validate \
  -input /opt/wirekite/license/wirekite.license \
  -keyfile /opt/wirekite/license/wirekite.key
```

### Usage Cap Exceeded (Exit Code 2)

Some licenses limit total bytes processed. When a cap is reached, the loader exits with code 2.

| Error Message                    | Cause                               |
| -------------------------------- | ----------------------------------- |
| `data migration limit exceeded`  | Data load byte cap reached          |
| `CDC replication limit exceeded` | Change replication byte cap reached |

Check current usage through the Web Interface or contact your license provider to increase the cap.

## Connection Issues

### DSN File Errors

| Error Message                                 | Cause                                    | Fix                                   |
| --------------------------------------------- | ---------------------------------------- | ------------------------------------- |
| `failed to open DSN file`                     | File path is wrong or file doesn't exist | Verify the path in your config        |
| `DSN file is empty or contains only comments` | DSN file has no connection string        | Add the connection string to the file |

### Database Connection Failures

| Error Message                          | Cause                        | Fix                                          |
| -------------------------------------- | ---------------------------- | -------------------------------------------- |
| `failed to open MySQL connection`      | Cannot connect to MySQL      | Check host, port, credentials                |
| `failed to open PostgreSQL connection` | Cannot connect to PostgreSQL | Check host, port, credentials                |
| `failed to connect to Spanner`         | Spanner connection failed    | Verify project ID, instance, and credentials |
| `failed to execute test query`         | Connected but query failed   | Check user permissions                       |

**Diagnosis steps:**

1. Test connectivity from the Wirekite host:
   ```bash theme={null}
   nc -zv <host> <port>
   ```

2. Test DNS resolution:
   ```bash theme={null}
   nslookup <hostname>
   ```

3. Use the Web Interface **Test Connection** button to validate credentials

4. Run a test query with [Wiretalk](/run/commandline):
   ```bash theme={null}
   wiretalk -dsn /path/to/dsn -sql "SELECT 1"
   ```

## Data Migration Issues

### Migration Stops Mid-Process

**Symptoms:** No new data arriving at target, logs show no recent activity.

**Diagnosis:**

```sql theme={null}
-- Check for incomplete operations on target
SELECT source, operation, mark, start_time, restarts
FROM wirekite.wirekite_progress
WHERE finish_time IS NULL
ORDER BY start_time DESC;
```

```bash theme={null}
-- Check if the process is still running
ps aux | grep -E "extractor|loader|orchestrator"

-- Check disk space
df -h
```

**Recovery:** Restart the migration. Wirekite automatically resumes from the last saved position using the [wirekite\_progress](/run/operations) table.

### Duplicate Key Errors During Load

**Cause:** Table already has data from a previous partial load, and the loader is re-inserting rows.

**Recovery:**

1. Check if the file was already loaded:
   ```sql theme={null}
   SELECT * FROM wirekite.wirekite_progress
   WHERE source = 'schema.table.0.dkt';
   ```

2. If the record shows `finish_time IS NOT NULL`, the file completed successfully and should be skipped automatically. If the recovery table is missing the record, truncate the target table and reset:
   ```sql theme={null}
   TRUNCATE TABLE target_schema.table_name;
   DELETE FROM wirekite.wirekite_progress
   WHERE target LIKE '%table_name%' AND operation = 'D';
   ```

### Schema Mismatch

**Symptoms:** Data load fails with type mismatch or column not found errors.

**Cause:** Source schema changed after schema extraction, or type mappings are incompatible.

**Recovery:**

1. Re-run schema extraction to capture the latest source schema
2. Verify type mappings in the generated SQL
3. Regenerate and re-apply the create table SQL on the target

### Out of Disk Space

**Symptoms:** File operations fail with `no space left on device`.

**Recovery:**

1. Free disk space (check `/tmp`, log directories, old data files)
2. Enable `removeFiles=true` in the mover and loader to clean up processed files
3. Restart the migration - it resumes from the recovery checkpoint

## CDC/Replication Issues

### MySQL / MariaDB / SingleStore

| Error                           | Cause                                       | Fix                                                                  |
| ------------------------------- | ------------------------------------------- | -------------------------------------------------------------------- |
| `failed to get binlog position` | Binary logging not enabled or binlog purged | Enable with `SET GLOBAL binlog_format='ROW'` and ensure `log_bin=ON` |
| `failed to start replication`   | Replication user lacks permissions          | Grant `SELECT, REPLICATION SLAVE, REPLICATION CLIENT`                |
| `failed to flush binary logs`   | Insufficient privileges                     | Grant `RELOAD` privilege                                             |

**Check binlog status:**

```sql theme={null}
SHOW VARIABLES LIKE 'log_bin';
SHOW MASTER STATUS;
```

### PostgreSQL / YugaByte / TigerData / AlloyDB

| Error                               | Cause                                               | Fix                                                   |
| ----------------------------------- | --------------------------------------------------- | ----------------------------------------------------- |
| `failed to create replication slot` | `max_replication_slots` reached or slot name in use | Increase `max_replication_slots` or drop unused slots |
| `failed to drop replication slot`   | Slot has an active consumer                         | Terminate the consuming connection first              |
| `Replication slot does not exist`   | Slot was dropped or never created                   | Recreate the replication slot                         |

**Check replication status:**

```sql theme={null}
-- List all slots
SELECT slot_name, active, restart_lsn FROM pg_replication_slots;

-- Check WAL level
SHOW wal_level;  -- Must be 'logical'

-- Check replication lag
SELECT slot_name,
       pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS lag_bytes
FROM pg_replication_slots;
```

**Drop a stuck slot:**

```sql theme={null}
SELECT pg_drop_replication_slot('wirekite_replication_slot');
```

### Oracle

| Error              | Cause                            | Fix                                        |
| ------------------ | -------------------------------- | ------------------------------------------ |
| LogMiner errors    | Supplemental logging not enabled | `ALTER DATABASE ADD SUPPLEMENTAL LOG DATA` |
| Archive log errors | Database not in ARCHIVELOG mode  | Switch to ARCHIVELOG mode                  |

### SQL Server

| Error                     | Cause                                 | Fix                                       |
| ------------------------- | ------------------------------------- | ----------------------------------------- |
| CDC not capturing changes | CDC not enabled on database or tables | Enable CDC on the database and each table |
| Transaction log full      | Log space exhausted                   | Increase log file size or run log backup  |

**Enable CDC:**

```sql theme={null}
-- Enable on database
EXEC sys.sp_cdc_enable_db;

-- Enable on table
EXEC sys.sp_cdc_enable_table
  @source_schema = 'dbo',
  @source_name = 'your_table',
  @role_name = NULL;
```

## Cloud Staging Issues

### S3/GCS Upload Failures

| Error                   | Cause                                  | Fix                                    |
| ----------------------- | -------------------------------------- | -------------------------------------- |
| `Upload error`          | Credentials invalid or expired         | Refresh AWS/GCS credentials            |
| `InitGCSBucketInfo err` | Bucket doesn't exist or not accessible | Verify bucket name and IAM permissions |

**Diagnosis:**

* AWS: Verify credentials with `aws sts get-caller-identity`
* GCS: Verify credentials with `gcloud auth list`
* Check bucket access and write permissions
* Verify the bucket region matches your configuration

### Snowflake Stage Errors

**Symptoms:** PUT or COPY commands fail.

**Check:**

1. Verify warehouse is running and has available compute
2. Check that the connection string specifies a valid warehouse
3. Verify the user has USAGE on the warehouse and stage

## Recovery Table Issues

### `wirekite_action must have 1 row in it`

**Cause:** The `wirekite_action` table was accidentally emptied or has extra rows.

**Fix:**

```sql theme={null}
-- Check current state
SELECT * FROM wirekite.wirekite_action;

-- Reset to exactly one row
DELETE FROM wirekite.wirekite_action;
INSERT INTO wirekite.wirekite_action VALUES ('RUN');
```

### High Restart Counts

If the `restarts` column in `wirekite_progress` shows high values for a file, the same operation is crashing repeatedly.

**Diagnosis:**

```sql theme={null}
SELECT source, operation, restarts, start_time
FROM wirekite.wirekite_progress
WHERE restarts > 3
ORDER BY restarts DESC;
```

Investigate the component logs for the specific file to find the root cause before restarting again.

## UX Server Issues

### Cannot Connect to UX Server

1. Verify the service is running: `sudo systemctl status wirekite-ux`
2. Check logs: `tail -f /opt/wirekite/ux/logs/ux-server.log`
3. Ensure port 8443 is open: `sudo lsof -i :8443`
4. Verify SSL certificates exist at `/opt/wirekite/ux/server/certs/`

### Authentication Issues

1. Clear browser cookies and retry
2. Check `users.json` in the config directory
3. Restart the UX server to clear all sessions: `sudo systemctl restart wirekite-ux`

## Diagnostic Tools

### Wiretalk

Run ad-hoc queries against any configured source or target to diagnose connectivity and data issues. See [Wiretalk](/run/commandline).

### TableValidator

Validate data integrity between source and target after migration. See [TableValidator](/run/validation).

### Web Interface Diagnostics

The [Web Interface](/run/execution/ux) provides:

* **Test Connection** - Validate database credentials
* **Download Logs** - ZIP archive of all migration logs
* **Real-time Progress** - Monitor extraction and loading rates
* **Validation** - Run data validation with configurable sample size

## Pre-Migration Checklist

Before starting a migration, verify:

<Steps>
  <Step title="Connectivity">
    Test connections to both source and target databases.
  </Step>

  <Step title="License">
    Validate your license is current and has sufficient usage cap.
  </Step>

  <Step title="Disk Space">
    Ensure adequate space on the Wirekite host for data files.
  </Step>

  <Step title="Replication Prerequisites">
    If using CDC, enable the required replication features on the source (binlog for MySQL, logical WAL for PostgreSQL, supplemental logging for Oracle, CDC for SQL Server).
  </Step>

  <Step title="Permissions">
    Verify database users have the required privileges for extraction and loading.
  </Step>

  <Step title="Schema">
    Extract and review the schema before starting data migration.
  </Step>
</Steps>