--merge option deletes all remote items in synced collection during import

[cyprian-nziim]

Describe the Bug

I was able to create a CICD flow to update my local schema an data to my remote instance when I push to my repo, however each time I modify the local data in a specific collection, all data in my remote instance is reset to my local instance data state deleting all the other items in the collection in my remote instance.

To Reproduce

I have a local directus instance (Installed Via NPM) and PostgreSQL. I do all my local work with this instance when I want to deploy I have the following file in my repo.

My Dockerfile

FROM directus/directus:11.2.1

USER root
RUN corepack enable

// Workaround to be able to install extensions with 'node' user, only required in v11.1.1
RUN rm node_modules/.modules.yaml

RUN pnpm install directus-extension-schema-sync@3.0.2


WORKDIR /directus

COPY ./schema-sync ./schema-sync

My docker-compose file

version: "8"
services:
  directus-app:
    build:
      context: .
      dockerfile: ./Dockerfile
    image: directus-app:1.0.6
    volumes:
      - ./schema-sync:/directus/schema-sync
      - ./extensions:/directus/extensions
    command: >
      sh -c "
        npx directus schema-sync import --merge
      "

Directus Version

v11.2.1

Hosting Strategy

Self-Hosted (Docker Image)

Database

PostgreSQL

1. Export local DB with "npx directus schema-sync export"
2. Perform an initial Commit and Push to repo
3. CICD builds the image and pushes to server, on startup runs "npx directus schema-sync import --merge" app is deployed all is good
4. Add a new item in remote instance to a collection you have marked for sync in "directus-app/schema-sync/directus_config.js"
5. Update or Add an Item in any collection via the directus interphase on the local instance, make sure its in a collection you have marked for sync in "directus-app/schema-sync/directus_config.js"
6. Run "npx directus schema-sync export" on local instance to sync changes
7. Commit and push to repo
8. CICD builds the image and pushes to server, on startup runs "npx directus schema-sync import --merge" this deletes all items in collection in remote instance and resets collection to local state

Version

3.0.2

Installed Extension Via

Docker file

[u12206050]

This is as intended. If you wish to exclude certain items from being synced(ie. Created, updated or deleted) then you need to update the filter in the query for that collection in the schema sync config.

[cyprian-nziim]

Hello @u12206050 , In this case, wouldn't you agree that the "--merge" option may not hold any significance?

[u12206050]

I agree something seems buggy here. Will have to investigate, however I don't use --merge myself, so havn't run into it before.

[u12206050]

@cyprian-nziim I reviewed the logic and code, and although I found some improvements that I made (3.0.5) the logic surrounding merge seems to work correctly.
However if you are using the getKey in config, and that results in conflicts (ie. duplicates), then these duplicates will be deleted, could that be the issue?

[cyprian-nziim]

@u12206050 Thanks for getting back to me. I think it's highly unlikely that an item could be created with the same UUID locally and online at the same time. If that's the case, then it's working as expected.

However, on several occasions, I have created many more items in the collection marked for sync online than there are locally in the same collection. So if there are any UUID conflicts, alot more items will be left untouched in the target collection online.

But, after deployment, everything in the target collection online is deleted. Whereas it has to merge the local data into the target collection online, overwrite any items with UUID conflicts and leave the rest untouched.

Were you able to reproduce the issue?
You can even try it with two local directus instances using separate db's.