Skip to content

Multiframe scrubbing improvements#397

Draft
BryonLewis wants to merge 26 commits into
masterfrom
multiframe-scrubbing-improvements
Draft

Multiframe scrubbing improvements#397
BryonLewis wants to merge 26 commits into
masterfrom
multiframe-scrubbing-improvements

Conversation

@BryonLewis

@BryonLewis BryonLewis commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Resolves #126

Summary

Improves multiframe raster scrubbing by fixing RGB detection for sequential rasters, adding server-generated frame preview images, and crossfading from preview to tiles on the client for smoother frame transitions.

Done

Sentinel downloader (scripts/sentinelDownload/)

  • Supports creating multiframe single-image Sentinel raster datasets via --single-file (uses gdal_translate to append frames as subdatasets)
  • Additional CLI arguments for date range, cloud cover, output naming, and clip size
  • Generates an ingest JSON alongside clipped GeoTIFFs for direct import into Geoodatalytics

Dataset / RGB fixes

  • Default band:1 injection removed when no source_filter is supplied — this was preventing Boston ortho imagery from being recognized as RGB
  • band:1 is still applied when explicitly defined in the style spec
  • Multiframe rasters no longer render as grayscaleframe and band are extracted from the style JSON and passed as separate query parameters (outside style) so django-large-image can auto-detect RGB imagery

RasterFramePreview model

  • Foreign keys to LayerStyle and LayerFrame (unique per style/frame pair)
  • Stores width, height, georeferenced bounds, and an S3 preview image
  • Lifecycle status: creating | regenerating | complete | failed
  • style_fingerprint (sha256 of repr_style_configs()) guards against stale concurrent generation
  • Cascades on frame or layer style deletion; S3/MinIO object deleted via post_delete signal

uvdat/core/frame_previews/

  • raster_style.py - Builds django-large-image–compatible style objects from LayerStyle DB records; keeps frame/band outside style
  • preview_regeneration.py - Marks previews stale, supersedes pending tasks, enqueues Celery regeneration on style save/ingest
  • fingerprint.py - Computes style fingerprint for concurrency control
  • types.py - Shared FramePreviewData type

uvdat/core/tasks/frame_preview.py

  • Celery task generates per-frame preview PNGs via django-large-image thumbnails
  • Uses raster_style.py for styling; scales resolution to keep previews between ~1k–4k
  • Writes RasterFramePreview rows with bounds and dimensions
  • Updates TaskResult on completion (used for future client notification)

Preview regeneration on style save

  • LayerStyle create/update hooks call invalidate_and_enqueue_previews()
  • checks fingerprint hash of the style to determine if new previews need to be regenerated
  • Existing preview images are cleared immediately; rows marked regenerating
  • Ingest command also triggers preview generation for multiframe raster styles
  • Only status=complete previews with an image are serialized to the API

API

  • /layers/ and /layer-styles/ return multiframe_previews — ordered list of { bounds, width, height, url } per raster frame (presigned S3 URLs)
  • preview_status aggregated on layer and layer-style: ready | generating | notready (omitted when N/A)
  • Queryset annotations avoid N+1 Python aggregation

Client-side frame previews

Flow

  1. User changes frame → layerStore.updateLayersShown
  2. styleStore.updateLayerStyles
  3. framePreviewStore.showPreviewThenTiles
    • Hide tile layer (raster-opacity: 0)
    • Show georeferenced preview image
    • Wait for tile source to finish loading
    • Crossfade preview → tiles once loaded
    • Preload adjacent frames
      web/src/utils/framePreviewLayer.ts
  • upsertPreviewLayer — fetch preview URL, add/update MapLibre source + raster layer
  • hidePreviewLayer — set visibility to none during transition
  • removePreviewLayer / removePreviewLayersExcept — cleanup; keep adjacent preloaded frames
  • waitForRasterSourceLoaded — poll until tile source is ready (with timeout)
  • fadeRasterOpacities — animated crossfade between preview and tile layers
    web/src/store/framePreview.ts
  • prefetchLayerPreviews — preload all preview images for a layer
  • showPreviewThenTiles — main preview → tile transition orchestration
  • dismissPreviewForLayer — skip previews while editing layer styles; restore tile opacity
    Style editing integration
  • styleStore.setLayerStyleEditing dismisses previews when entering style edit mode
  • Previews re-enabled via updateLayerStyles after exiting edit mode
  • LayerStyle.vue watches active layer to toggle editing state

Tests

  • uvdat/core/tests/test_frame_preview.py — model serialization, API fields, regeneration hooks, fingerprint guards, raster style helpers

TODO

Client refresh after style save

  • Clear stale multiframe_previews from client state immediately on style save
  • Respect preview_status on the client — fall back to tiles while generating
  • WebSocket handler for task_type: "frame_preview" completion → refetch layer style, prefetch new previews, re-enable transitions
  • Optional UX: subtle "Updating frame previews…" indicator in the style panel

Coverage / polish

  • Test multiple styles on the same multiframe layer (switching between saved styles)
  • Add node animation output layer to frame scrubbing (currently raster layers only)

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 5, 2026

Copy link
Copy Markdown

Deploying geodatalytics with  Cloudflare Pages  Cloudflare Pages

Latest commit: ff89a0b
Status: ✅  Deploy successful!
Preview URL: https://ffed0db9.geodatalytics.pages.dev
Branch Preview URL: https://multiframe-scrubbing-improve.geodatalytics.pages.dev

View logs

@BryonLewis BryonLewis force-pushed the multiframe-scrubbing-improvements branch from bb34a8b to aa64e06 Compare June 5, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Layer Frame Preloading

1 participant