fix: move JSON function creation into EF Core migrations to prevent startup failure on restart#1313
Conversation
…on restart On application restart the hosted service fails with: SqlException: There is already an object named 'json_exists' in the database. The #if RELEASE guard in JsonFunction.cs skips DROP FUNCTION IF EXISTS statements in release builds, so CREATE FUNCTION fails if the functions already exist from a previous run. Replace the DROP + CREATE pattern with idempotent alternatives: - SQL Server: CREATE OR ALTER FUNCTION (supported since SQL Server 2016 SP1) - MySQL: CREATE OR REPLACE FUNCTION PostgreSQL was already using CREATE OR REPLACE FUNCTION correctly. DROP FUNCTION IF EXISTS statements are removed from both SQL files.
|
Does not seem to work for MySQL According to claude something like htis could work EDIT: This does not solve the problem, either ...perhaps we just ignore the exception |
|
My Bad.. It works for Postgres and SQL Server - which i have tested, but i do not have a MySQL instance to test on.. Shall we revert the MySQL on for now and i can have a look and spin up a MySQL container? |
|
i have a different idea for this, how about we add the functions to the code first migrations? |
- Replace SqlDialectInitializer startup logic with proper EF Core migrations for all three providers (MySQL, SQL Server, Postgres) - SQL Server: uses CREATE OR ALTER FUNCTION (idempotent, no DROP needed) - MySQL: uses DROP FUNCTION IF EXISTS + CREATE FUNCTION in migration - Remove SqlDialectInitializer registration from production ServiceExtensions - Add migration tests: idempotency and upgrade-from-pre-migration-database
|
I think you also have to remove the existing initialization logic. And what happens when you already have these functions? |
…n test fixtures - Test fixtures now use the same code path as production (MigrateAsync) - DatabaseCreator and SqlDialectInitializer are no longer needed and deleted - Functions are created via the AddJsonFunctions migration, not at every startup
|
Looks good. There is just the reduncant code in Dialect and SqlFunctions (initialize stuff) |
JsonFunction.InitializeAsync() and Dialect.InitializeAsync() — still have a purpose. They're called on line ~125 in each migration test file to simulate the old startup behavior in the Should_migrate_when_functions_already_exist regression test. I think we still need these tests as it makes sure that there is a solid upgrade path from a system where there were no code first migrations. |
|
Yesterday, before you updated PR for migrations I asked the AI to provide a solution and it came up with some clever ideas like the JsonFunctionmigration class |
|
i will have to leave that one in your capable hands if you want to change the way its doing it. I think the migrations should be good in the meantime :) |
|
No, it is implementation is fine, but it does not work yet. I think you also have to register the migration for the content database in the tests |
…prefix migration history
Two bugs fixed in the EF Core test fixtures (PostgresFixture, MySqlFixture, SqlServerFixture):
1. Replace DatabaseMigrator with EnsureCreatedAsync + Dialect.InitializeAsync
TestDbContext* are test-only contexts with no EF migration files, so
DatabaseMigrator<TestDbContext*>.InitializeAsync called MigrateAsync which was
a complete no-op — no tables were ever created and all integration tests failed
with 'relation does not exist'.
EnsureCreatedAsync builds the schema directly from the EF Core model, which is
the correct approach for contexts without migrations. Dialect.InitializeAsync is
then called explicitly to create the database-specific JSON functions (json_exists
etc.) that EnsureCreated does not set up.
2. Add per-prefix MigrationsHistoryTable for named ContentDbContext registrations
DynamicTables.PrepareAsync calls MigrateAsync on the named ContentDbContext
(e.g. PostgresContentDbContext) to create per-app/schema dedicated tables such
as '__c5_ContentsAll'. The migration (AddInitial) reads TableName.Prefix to
build the table name at runtime.
All named contexts shared the default '__EFMigrationsHistory' table, so after
the first prefix ran AddInitial and recorded it, every subsequent prefix saw the
migration as already applied and skipped it — leaving its dedicated tables
uncreated and causing 'relation __cN_ContentsAll does not exist' failures in
all but the first dedicated-table test.
Setting options.MigrationsHistoryTable(\$"{name}MigrationHistory") gives each
prefix its own independent migration history, so AddInitial runs once per prefix
and creates the correct tables each time.
Also add *.lscache to .gitignore (C# language server cache files).
Problem
On restart, Squidex would crash with:
Root cause:
JsonFunction.cshad a#if RELEASEguard that skipped theDROP FUNCTION IF EXISTSstatements in release builds, soCREATE FUNCTIONwould fail on second startup because the functions already existed.Fix
Rather than patching the startup DROP/CREATE logic, JSON function creation is now a proper EF Core migration — it runs exactly once per database and is skipped on subsequent restarts via EF's
__EFMigrationsHistorytable.Per-provider approach
CREATE OR ALTER FUNCTIONDROP FUNCTION IF EXISTS+CREATE FUNCTIONCREATE OR REPLACE FUNCTION(that's MariaDB)CREATE OR REPLACE FUNCTIONChanges
20260512000000_AddJsonFunctionsmigration for MySQL20260512000001_AddJsonFunctionsmigration for SQL Server20260512000002_AddJsonFunctionsmigration for PostgreSQLSqlDialectInitializer<TContext>from production DI (it remains available for test fixtures that useEnsureCreatedinstead ofMigrate)#if RELEASEguard fromMySql/JsonFunction.cs(still used by test fixtures)json_function.sql: allCREATE FUNCTIONchanged toCREATE OR ALTER FUNCTION,DROPlines removedUpgrade safety
Existing databases that have functions created by the old
SqlDialectInitializer(no migration history entry) are handled safely:CREATE OR ALTER FUNCTIONsucceeds regardless of whether the function already existsDROP IF EXISTS+CREATEis safe even if functions already existThis is covered by the
Should_migrate_when_functions_already_existregression tests.How to add/update JSON functions in future
If a JSON helper function needs to be changed or a new one added, do not modify the existing migration files. Instead:
1. Update the SQL file
Edit the relevant
json_function.sqlfor the provider(s) you're changing:backend/src/Squidex.Data.EntityFramework/Providers/MySql/json_function.sqlbackend/src/Squidex.Data.EntityFramework/Providers/SqlServer/json_function.sqlbackend/src/Squidex.Data.EntityFramework/Providers/Postgres/json_function.sql2. Create a new migration
Create a new migration file with a timestamp later than
20260512. For example, to update a MySQL function:You'll also need a
.Designer.csfile — copy the nearest existing one and update the[Migration("...")]attribute and class name to match.3. Update the model snapshot
Run the EF tooling to regenerate the model snapshot, or copy-edit
*DbContextModelSnapshot.csto include the new migration ID in its[DbContext]annotation.