Skip to content

fix: move JSON function creation into EF Core migrations to prevent startup failure on restart#1313

Merged
SebastianStehle merged 6 commits into
Squidex:masterfrom
RTJoe:fix/sql-create-or-alter-functions
May 17, 2026
Merged

fix: move JSON function creation into EF Core migrations to prevent startup failure on restart#1313
SebastianStehle merged 6 commits into
Squidex:masterfrom
RTJoe:fix/sql-create-or-alter-functions

Conversation

@RTJoe
Copy link
Copy Markdown
Contributor

@RTJoe RTJoe commented May 12, 2026

Problem

On restart, Squidex would crash with:

SqlException: There is already an object named 'json_exists' in the database.

Root cause: JsonFunction.cs had a #if RELEASE guard that skipped the DROP FUNCTION IF EXISTS statements in release builds, so CREATE FUNCTION would fail on second startup because the functions already existed.

Fix

Rather than patching the startup DROP/CREATE logic, JSON function creation is now a proper EF Core migration — it runs exactly once per database and is skipped on subsequent restarts via EF's __EFMigrationsHistory table.

Per-provider approach

Provider SQL used Why
SQL Server CREATE OR ALTER FUNCTION Natively idempotent — works whether functions exist or not
MySQL DROP FUNCTION IF EXISTS + CREATE FUNCTION MySQL 8 has no CREATE OR REPLACE FUNCTION (that's MariaDB)
PostgreSQL CREATE OR REPLACE FUNCTION Supported natively

Changes

  • Added 20260512000000_AddJsonFunctions migration for MySQL
  • Added 20260512000001_AddJsonFunctions migration for SQL Server
  • Added 20260512000002_AddJsonFunctions migration for PostgreSQL
  • Removed SqlDialectInitializer<TContext> from production DI (it remains available for test fixtures that use EnsureCreated instead of Migrate)
  • Removed #if RELEASE guard from MySql/JsonFunction.cs (still used by test fixtures)
  • SQL Server json_function.sql: all CREATE FUNCTION changed to CREATE OR ALTER FUNCTION, DROP lines removed

Upgrade safety

Existing databases that have functions created by the old SqlDialectInitializer (no migration history entry) are handled safely:

  • SQL Server: CREATE OR ALTER FUNCTION succeeds regardless of whether the function already exists
  • MySQL: DROP IF EXISTS + CREATE is safe even if functions already exist

This is covered by the Should_migrate_when_functions_already_exist regression tests.


How to add/update JSON functions in future

If a JSON helper function needs to be changed or a new one added, do not modify the existing migration files. Instead:

1. Update the SQL file

Edit the relevant json_function.sql for the provider(s) you're changing:

  • backend/src/Squidex.Data.EntityFramework/Providers/MySql/json_function.sql
  • backend/src/Squidex.Data.EntityFramework/Providers/SqlServer/json_function.sql
  • backend/src/Squidex.Data.EntityFramework/Providers/Postgres/json_function.sql

2. Create a new migration

Create a new migration file with a timestamp later than 20260512. For example, to update a MySQL function:

// backend/src/Squidex.Data.EntityFramework/Providers/MySql/App/Migrations/20270101000000_UpdateMyFunction.cs
namespace Squidex.Providers.MySql.App.Migrations;

public partial class UpdateMyFunction : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.Sql("DROP FUNCTION IF EXISTS json_my_function;;", suppressTransaction: true);
        migrationBuilder.Sql("""
            CREATE FUNCTION json_my_function(...) RETURNS ...
            BEGIN
              ...
            END;;
            """, suppressTransaction: true);
    }

    protected override void Down(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.Sql("DROP FUNCTION IF EXISTS json_my_function;;", suppressTransaction: true);
    }
}

You'll also need a .Designer.cs file — copy the nearest existing one and update the [Migration("...")] attribute and class name to match.

Note for SQL Server: omit suppressTransaction: true and use CREATE OR ALTER FUNCTION so no DROP is needed.

3. Update the model snapshot

Run the EF tooling to regenerate the model snapshot, or copy-edit *DbContextModelSnapshot.cs to include the new migration ID in its [DbContext] annotation.

…on restart

On application restart the hosted service fails with:
  SqlException: There is already an object named 'json_exists' in the database.

The #if RELEASE guard in JsonFunction.cs skips DROP FUNCTION IF EXISTS
statements in release builds, so CREATE FUNCTION fails if the functions
already exist from a previous run.

Replace the DROP + CREATE pattern with idempotent alternatives:
- SQL Server: CREATE OR ALTER FUNCTION (supported since SQL Server 2016 SP1)
- MySQL:      CREATE OR REPLACE FUNCTION

PostgreSQL was already using CREATE OR REPLACE FUNCTION correctly.
DROP FUNCTION IF EXISTS statements are removed from both SQL files.
@SebastianStehle
Copy link
Copy Markdown
Contributor

SebastianStehle commented May 12, 2026

Does not seem to work for MySQL

According to claude something like htis could work

SELECT GET_LOCK('db_migrations', 10); -- waits up to 10s

DROP FUNCTION IF EXISTS json_empty;
CREATE FUNCTION json_empty(...) ...;

SELECT RELEASE_LOCK('db_migrations');

EDIT: This does not solve the problem, either ...perhaps we just ignore the exception

@RTJoe
Copy link
Copy Markdown
Contributor Author

RTJoe commented May 12, 2026

My Bad.. It works for Postgres and SQL Server - which i have tested, but i do not have a MySQL instance to test on..

Shall we revert the MySQL on for now and i can have a look and spin up a MySQL container?

@RTJoe
Copy link
Copy Markdown
Contributor Author

RTJoe commented May 12, 2026

i have a different idea for this, how about we add the functions to the code first migrations?

- Replace SqlDialectInitializer startup logic with proper EF Core migrations
  for all three providers (MySQL, SQL Server, Postgres)
- SQL Server: uses CREATE OR ALTER FUNCTION (idempotent, no DROP needed)
- MySQL: uses DROP FUNCTION IF EXISTS + CREATE FUNCTION in migration
- Remove SqlDialectInitializer registration from production ServiceExtensions
- Add migration tests: idempotency and upgrade-from-pre-migration-database
@RTJoe RTJoe changed the title fix: use CREATE OR ALTER/REPLACE FUNCTION to prevent startup failure on restart fix: move JSON function creation into EF Core migrations to prevent startup failure on restart May 12, 2026
@SebastianStehle
Copy link
Copy Markdown
Contributor

I think you also have to remove the existing initialization logic. And what happens when you already have these functions?

RTJoe added 3 commits May 13, 2026 08:57
…n test fixtures

- Test fixtures now use the same code path as production (MigrateAsync)
- DatabaseCreator and SqlDialectInitializer are no longer needed and deleted
- Functions are created via the AddJsonFunctions migration, not at every startup
@SebastianStehle
Copy link
Copy Markdown
Contributor

Looks good. There is just the reduncant code in Dialect and SqlFunctions (initialize stuff)

@RTJoe
Copy link
Copy Markdown
Contributor Author

RTJoe commented May 13, 2026

Looks good. There is just the reduncant code in Dialect and SqlFunctions (initialize stuff)

JsonFunction.InitializeAsync() and Dialect.InitializeAsync() — still have a purpose. They're called on line ~125 in each migration test file to simulate the old startup behavior in the Should_migrate_when_functions_already_exist regression test.

I think we still need these tests as it makes sure that there is a solid upgrade path from a system where there were no code first migrations.

@SebastianStehle
Copy link
Copy Markdown
Contributor

Yesterday, before you updated PR for migrations I asked the AI to provide a solution and it came up with some clever ideas like the JsonFunctionmigration class

#1314

@RTJoe
Copy link
Copy Markdown
Contributor Author

RTJoe commented May 14, 2026

i will have to leave that one in your capable hands if you want to change the way its doing it.

I think the migrations should be good in the meantime :)

@SebastianStehle
Copy link
Copy Markdown
Contributor

No, it is implementation is fine, but it does not work yet. I think you also have to register the migration for the content database in the tests

…prefix migration history

Two bugs fixed in the EF Core test fixtures (PostgresFixture, MySqlFixture, SqlServerFixture):

1. Replace DatabaseMigrator with EnsureCreatedAsync + Dialect.InitializeAsync

   TestDbContext* are test-only contexts with no EF migration files, so
   DatabaseMigrator<TestDbContext*>.InitializeAsync called MigrateAsync which was
   a complete no-op — no tables were ever created and all integration tests failed
   with 'relation does not exist'.

   EnsureCreatedAsync builds the schema directly from the EF Core model, which is
   the correct approach for contexts without migrations. Dialect.InitializeAsync is
   then called explicitly to create the database-specific JSON functions (json_exists
   etc.) that EnsureCreated does not set up.

2. Add per-prefix MigrationsHistoryTable for named ContentDbContext registrations

   DynamicTables.PrepareAsync calls MigrateAsync on the named ContentDbContext
   (e.g. PostgresContentDbContext) to create per-app/schema dedicated tables such
   as '__c5_ContentsAll'. The migration (AddInitial) reads TableName.Prefix to
   build the table name at runtime.

   All named contexts shared the default '__EFMigrationsHistory' table, so after
   the first prefix ran AddInitial and recorded it, every subsequent prefix saw the
   migration as already applied and skipped it — leaving its dedicated tables
   uncreated and causing 'relation __cN_ContentsAll does not exist' failures in
   all but the first dedicated-table test.

   Setting options.MigrationsHistoryTable(\$"{name}MigrationHistory") gives each
   prefix its own independent migration history, so AddInitial runs once per prefix
   and creates the correct tables each time.

Also add *.lscache to .gitignore (C# language server cache files).
@SebastianStehle SebastianStehle merged commit 8a4cae1 into Squidex:master May 17, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants