Skip to content

Fix retention coverage across group-scoped and collaborative conversations #1054

Description

@paullizer

Issue

Retention policy execution does not consistently map conversation ownership to the current conversation storage model.

Current behavior has these gaps:

  • group-single-user conversations are stored in conversations and are processed by the creator's personal retention policy, even though group retention must govern them.
  • personal_multi_user and group_multi_user conversations are stored in collaboration_conversations, use updated_at, and are not selected by the existing retention implementation.
  • Group retention currently queries only the legacy group_conversations container.
  • Newly created group documents omit retention_policy; missing values resolve to organization defaults at runtime, but the intended default state is not persisted explicitly.

Steps to Reproduce

  1. Configure different personal and group conversation-retention defaults.
  2. Create a private group-single-user conversation, a personal_multi_user conversation, and a group_multi_user conversation.
  3. Age their applicable activity timestamps beyond the configured retention periods.
  4. Execute personal and group retention.
  5. Observe that the private group conversation follows personal retention and the collaboration conversations are not processed.

Expected Behavior

Conversation retention must follow this ownership matrix regardless of the backing Cosmos container:

  • group-single-user: group retention.
  • personal_multi_user: the creator's personal retention.
  • group_multi_user: group retention.
  • Legacy conversations in group_conversations: group retention.
  • Newly created groups initialize retention_policy.conversation_retention_days and retention_policy.document_retention_days to "default".

Actual Behavior

The retention implementation selects personal conversations from conversations by user_id and group conversations only from group_conversations by group_id. It does not account for conversation type and scope across conversations and collaboration_conversations, nor the latter container's updated_at timestamp.

Impact

Group retention policy is not reliably enforced. Group-scoped private conversations can follow the wrong policy, while shared personal and shared group conversations can be retained indefinitely. This is a P0 data-governance and compliance gap.

Acceptance Criteria

  • group-single-user conversations are evaluated against the retention policy of their primary group, even though they are stored in conversations.
  • group-single-user conversations are excluded from personal conversation retention to prevent conflicting or duplicate policy execution.
  • personal_multi_user conversations are evaluated against the creating user's personal retention policy.
  • group_multi_user conversations are evaluated against the policy for scope.group_id.
  • Legacy group_conversations records continue to use group retention.
  • New groups persist both retention values as "default" without changing the behavior of existing groups that omit retention_policy.
  • Retention supports both last_updated and collaboration updated_at activity fields without deleting records that lack a valid timestamp.
  • Collaboration deletion or archival handles associated messages, per-user state, linked legacy source records, generated/blob-backed files, activity logging, and conversation-cache invalidation without leaving orphaned data.
  • Converted conversations are not counted or deleted twice through hidden source and collaboration records.
  • Focused regression tests cover every row in the policy matrix, explicit custom policies, organization defaults, none, missing timestamps, and already-deleted race handling.
  • Fix documentation and application version metadata are updated consistently.

Notes

This is a follow-up to #585, which introduced group/public retention settings. The gap arose as conversation storage expanded across legacy, personal single-user, and collaboration containers.

Likely implementation areas:

  • application/single_app/functions_retention_policy.py
  • application/single_app/functions_group.py
  • application/single_app/functions_collaboration.py
  • retention functional tests under functional_tests/

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingsecurity_improvementThis issue results in an improvement to security

Type

No type

Projects

Status
Released

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions