Skip to content

Conversation

@CodingAnarchy
Copy link

Summary

This PR implements template database cloning for sqlx::test to significantly speed up test runs when using migrations. Instead of running migrations for every test, a template database is created once and cloned for each test.

Before (slow)

For each test:
  CREATE DATABASE test_xyz
  → Run all migrations (slow!)
  → Load fixtures
  → Run test
  → DROP DATABASE

After (fast)

Once per migration set:
  CREATE DATABASE _sqlx_template_<hash>
  → Run all migrations

For each test:
  Clone template to test_xyz (fast!)
  → Load fixtures
  → Run test
  → DROP DATABASE

Implementation Details

Core Infrastructure (sqlx-core/src/testing/mod.rs)

  • Add migrations_hash() - computes SHA256 hash of all migration checksums for template naming
  • Add template_db_name() - generates template database name from hash
  • Extend TestContext with from_template: bool field
  • Modify setup_test_db() to skip migrations when cloned from template

MySQL Implementation (sqlx-mysql/src/testing/mod.rs)

  • Add _sqlx_test_templates tracking table for template metadata
  • Use GET_LOCK() for synchronization during template creation
  • Clone via mysqldump/mysql commands (fast) with in-process SQL fallback
  • Handle existing template databases from previous runs (idempotent)
  • Compatible with both MySQL and MariaDB

Configuration

  • Opt-out: Set SQLX_TEST_NO_TEMPLATE=1 to disable template cloning
  • Template names include migration hash, so changing migrations automatically creates new templates

Test Plan

  • Unit tests for migrations_hash() and template_db_name() functions
  • Integration tests verifying template creation and reuse
  • Integration tests verifying fixtures work with cloned templates
  • CI passes on all MySQL versions (8.4, 8.0, 5.7)
  • CI passes on all MariaDB versions (verylatest, 11.8, 11.4, 10.11, 10.6)
  • CI passes on all runtime/TLS combinations

Future Work

  • PostgreSQL implementation using native CREATE DATABASE ... TEMPLATE (follow-up PR)
  • SQLite implementation using file copy (follow-up PR)

🤖 Generated with Claude Code

CodingAnarchy and others added 6 commits December 19, 2025 15:31
Add template database cloning optimization to avoid running migrations for
every test. When multiple tests use the same migrations, a template database
is created once and cloned for each test, significantly speeding up test runs.

Implementation details:
- Add migrations_hash() to compute SHA256 of migration checksums
- Add template_db_name() to generate template database names
- Extend TestContext with from_template field to track cloning
- Modify setup_test_db() to skip migrations when cloned from template
- MySQL: Use mysqldump/mysql for fast cloning with in-process fallback
- Add _sqlx_test_templates tracking table with GET_LOCK synchronization
- Add SQLX_TEST_NO_TEMPLATE env var to opt out of template cloning
- Add comprehensive tests for template functionality
- Add template tests to MySQL and MariaDB CI jobs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use CREATE DATABASE IF NOT EXISTS for idempotent template creation.
Check if migrations already exist before running them, allowing reuse
of template databases that exist but weren't registered in the tracking
table (e.g., from a previous CI run or interrupted process).
Use INSERT IGNORE when registering templates to handle race conditions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…chema

Query _sqlx_migrations table directly to check if migrations exist.
This handles the case where the table exists with entries from a
previous run more reliably than checking information_schema.tables.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Check GET_LOCK returns 1 (success) before proceeding
- Add debug output to show migration count or error during template check
- This should help diagnose the race condition in CI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
MariaDB returns NULL when GET_LOCK is called with -1 timeout.
Use 300 second timeout instead for cross-database compatibility.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
GitHub has deprecated the macOS-13 runner. Remove it from the
sqlx-cli workflow matrices.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant