Skip to content

Schema Alignment Plan#352

Merged
Yostra merged 1 commit intomainfrom
schema_update
May 5, 2026
Merged

Schema Alignment Plan#352
Yostra merged 1 commit intomainfrom
schema_update

Conversation

@Yostra
Copy link
Copy Markdown
Collaborator

@Yostra Yostra commented Apr 29, 2026

Stripe Data Schema Projection Alignment Plan


Requirements To Align

  • Table names are pluralized. Requirement says singular snake_case table names; code uses customers, payment_intents, v2_core_accounts.
  • Expandable reference metadata is dropped. OpenAPI parser marks references, but catalog conversion loses it, so references may become JSON instead of ID text columns.
  • Deletes are hard, drop rows. (done in separate PR)
  • Metadata naming differs. rename _last_synced_at to _synced_at and implement it to be the timestamp of the last time row was written to the db/sheet

Stopped and left to be done as a part of the follow-on PR. (The diff grew to a size that is really hard to review)

  • V2 date-time fields become text. Requirement says v2 timestamps should use native date/time types.
  • List envelopes may stay inline. Requirement says {data, has_more, url, object: "list"} fields should be queried through child tables, not stored on parent rows.

@Yostra Yostra force-pushed the schema_update branch 2 times, most recently from 0a7a318 to dba8949 Compare May 1, 2026 21:55
@Yostra Yostra marked this pull request as ready for review May 1, 2026 22:46
@Yostra Yostra assigned Yostra and unassigned Yostra May 1, 2026
@Yostra Yostra requested a review from kdhillon-stripe May 2, 2026 00:18
// Map known prefixes to Stripe product groups (singular per resource naming spec).
const REFINEMENTS: Record<string, string> = {
subscription: 'Billing',
subscriptions: 'Billing',
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not blocking: we should remove this, this should not be manually defined

// `_updated_at` is hardcoded below; upsertMany writes it (DDR-009).
if (name === '_updated_at') continue
// System columns are hardcoded below; upsertMany writes both.
if (name === '_updated_at' || name === '_synced_at') continue
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the difference between synced at and updated at? we should remove updated at if its just a duplicate

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_synced_at is the last time the record was seen by sync-engine, regardless of whether the record changed. It can be used to detect missed delete events or webhooks.

_updated_at represents the freshness of the record, to the best of our knowledge. It's used to determine the latest version of an object and determine if we should upsert or not.

We determine its value using the following priority order:

  1. The native updated field
  2. The event created timestamp, if the update came from a webhook.
  3. The Stripe response header time (during backfill).
  4. now() as a fallback.

Comment thread packages/openapi/specParser.ts
Comment on lines 6 to +10
export const OPENAPI_RESOURCE_TABLE_ALIASES: Record<string, string> = {
'radar.early_fraud_warning': 'early_fraud_warnings',
'entitlements.active_entitlement': 'active_entitlements',
'entitlements.feature': 'features',
item: 'checkout_session_line_items',
}

/**
* Compatibility columns that should exist even if not present in the current OpenAPI shape.
* This preserves backwards compatibility for existing queries and write paths.
* todo: Remove this
*/
export const OPENAPI_COMPATIBILITY_COLUMNS: Record<string, ParsedColumn[]> = {
active_entitlements: [
{ name: 'customer', type: 'text', nullable: true },
{ name: 'object', type: 'text', nullable: true },
{ name: 'feature', type: 'text', nullable: true },
{ name: 'livemode', type: 'boolean', nullable: true },
{ name: 'lookup_key', type: 'text', nullable: true },
],
checkout_session_line_items: [
{ name: 'checkout_session', type: 'text', nullable: true },
{ name: 'amount_discount', type: 'bigint', nullable: true },
{ name: 'amount_tax', type: 'bigint', nullable: true },
],
customers: [{ name: 'deleted', type: 'boolean', nullable: true }],
early_fraud_warnings: [{ name: 'payment_intent', type: 'text', nullable: true }],
features: [
{ name: 'object', type: 'text', nullable: true },
{ name: 'name', type: 'text', nullable: true },
{ name: 'lookup_key', type: 'text', nullable: true },
{ name: 'active', type: 'boolean', nullable: true },
{ name: 'livemode', type: 'boolean', nullable: true },
{ name: 'metadata', type: 'json', nullable: true },
],
subscription_items: [
{ name: 'deleted', type: 'boolean', nullable: true },
{ name: 'subscription', type: 'text', nullable: true },
],
'radar.early_fraud_warning': 'early_fraud_warning',
'entitlements.active_entitlement': 'active_entitlement',
'entitlements.feature': 'feature',
item: 'checkout_session_line_item',
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also remove these?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Comment thread packages/openapi/specParser.ts
@Yostra Yostra merged commit 822000f into main May 5, 2026
33 of 35 checks passed
@Yostra Yostra deleted the schema_update branch May 5, 2026 00:10
@Yostra Yostra mentioned this pull request May 6, 2026
tonyxiao pushed a commit that referenced this pull request May 7, 2026
Reverts back to 5024aac (change soft to hard delete).

Reverted commits:
- 921fe63 Add source and destination for custom object support (#356)
- 95eaaa9 chore: remove Vercel docs deploy CI jobs (#360)
- 822000f squash (#352)


Committed-By-Agent: claude

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants