Skip to content

Integrate webxdc WebView audio playback with Android media session#142

Draft
Copilot wants to merge 3 commits intomainfrom
copilot/integrate-webxdc-audio-playback
Draft

Integrate webxdc WebView audio playback with Android media session#142
Copilot wants to merge 3 commits intomainfrom
copilot/integrate-webxdc-audio-playback

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

Webxdc mini-apps playing audio have no Android system media notification, no background playback, and no hardware media key support. This adds full media session integration for webxdc audio — matching what Chrome/Firefox do for browser tab audio.

New: WebxdcMediaSessionService

A MediaSessionService backed by a stub SimpleBasePlayer (no ExoPlayer — the WebView owns the audio engine). The stub player exists solely to drive the system media notification and respond to play/pause from the notification or hardware keys.

  • Accepts 4 custom SessionCommands from WebxdcActivity: WEBXDC_AUDIO_STARTED (+ title/artist/msgId for notification tap intent), WEBXDC_AUDIO_STOPPED, WEBXDC_AUDIO_PAUSED, WEBXDC_AUDIO_RESUMED
  • Relays notification play/pause back to the activity via package-scoped broadcasts (ACTION_NOTIFICATION_PAUSE / ACTION_NOTIFICATION_RESUME)
  • Updates SessionActivity PendingIntent per-instance so tapping the notification reopens the correct webxdc

Modified: WebxdcActivity

  • Connects a MediaController to WebxdcMediaSessionService on onCreate; releases + sends AUDIO_STOPPED on onDestroy
  • Registers BroadcastReceiver for notification pause/resume → evaluateJavascript to pause/play <audio>/<video> in the WebView
  • Adds @JavascriptInterface methods (notifyAudioStarted, notifyAudioPaused, notifyAudioStopped, notifyAudioResumed) to InternalJSApi, all dispatched to main thread via Util.runOnMain
  • Overrides pauseWebViewOnPause()!isAudioPlaying to keep WebView JS timers and audio alive in the background

Modified: WebViewActivity

Adds pauseWebViewOnPause() hook (default true) so subclasses can suppress webView.onPause():

@Override
protected boolean pauseWebViewOnPause() {
    return !isAudioPlaying;
}

Modified: webxdc.js

Appended after the webxdc API setup — runs inside the iframe context where InternalJSApi is already accessible. Uses setInterval polling (2 s) to attach play/pause/ended listeners to <audio>/<video> elements including dynamically created ones:

el.addEventListener('play', function() {
  if (window.InternalJSApi) InternalJSApi.notifyAudioStarted(document.title || '');
});

Modified: AndroidManifest.xml

Registers WebxdcMediaSessionService with foregroundServiceType="mediaPlayback" alongside the existing AudioPlaybackService. Existing voice message playback is untouched.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • dl.google.com
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED -Xmx4608m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/REDACTED/.gradle/wrapper/dists/gradle-8.13-bin/5xuhj0ry160q40clulazy9h7d/gradle-8.13/lib/gradle-daemon-main-8.13.jar (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

Feature: Integrate Webxdc WebView audio playback with Android media session

When a webxdc mini-app plays music/audio, it should integrate with the Android system media notification (like how Chrome/Firefox browsers do on Android when media is playing in a tab), and the webxdc activity should be allowed to keep running in the background while audio is playing.

How it works in browsers (reference behavior)

When a browser tab plays audio, Android:

  1. Shows a media notification with play/pause controls
  2. Allows the audio to keep playing when the user navigates away (background)
  3. Responds to hardware media keys (play/pause, next, previous)
  4. Stops playback when another app requests audio focus

Existing infrastructure

The app already has AudioPlaybackService (src/main/java/org/thoughtcrime/securesms/service/AudioPlaybackService.java) — a MediaSessionService backed by ExoPlayer + MediaSession (androidx.media3) — used for voice message playback. The manifest already declares FOREGROUND_SERVICE_MEDIA_PLAYBACK permission and registers AudioPlaybackService.

What needs to be implemented

The approach is to create a new dedicated WebxdcMediaSessionService (modeled after AudioPlaybackService) that uses a MediaSession with no ExoPlayer (the actual audio is played by the WebView's internal audio engine, not ExoPlayer). Instead it uses MediaSession purely to:

  • Post the system media notification (managed automatically by MediaSessionService)
  • Register Player.Listener / MediaSession.Callback so the notification's play/pause button works (by calling back into the WebView via JS)
  • Keep the process alive in the foreground while audio is playing

Files to create

src/main/java/org/thoughtcrime/securesms/service/WebxdcMediaSessionService.java

A new MediaSessionService that:

  • Holds a MediaSession wrapping a stub Player (a ForwardingSimpleBasePlayer or a minimal SimpleBasePlayer subclass from androidx.media3 that reports STATE_READY/STATE_IDLE and playWhenReady).
    • Use androidx.media3.common.SimpleBasePlayer (available in media3 1.x): override getState() to report the current playback state, and handleSetPlayWhenReady()/handleStop() to relay commands back to WebxdcActivity via a broadcast or a bound service interface.
  • Exposes custom SessionCommands:
    • "WEBXDC_AUDIO_STARTED" (args: title, artist/app-name, optional artwork as byte[]) — called by WebxdcActivity when the WebView reports audio playing; updates the MediaMetadata on the session and puts the service into foreground (starts it).
    • "WEBXDC_AUDIO_STOPPED" — called by WebxdcActivity when audio stops; removes the notification and stops the foreground service.
    • "WEBXDC_AUDIO_PAUSED" — updates player state to paused.
    • "WEBXDC_AUDIO_RESUMED" — updates player state to playing.
  • The stub player's handleSetPlayWhenReady(false) (user pressed pause in notification) should send a local broadcast or callback that WebxdcActivity listens to, calling webView.evaluateJavascript(...) to pause the WebView audio (e.g. document.querySelectorAll('audio,video').forEach(el => el.pause())).
  • The stub player's handleSetPlayWhenReady(true) should resume playback in the WebView similarly.
  • Sets sessionActivity PendingIntent to reopen the WebxdcActivity (the activity provides msgId, accountId extras via "WEBXDC_AUDIO_STARTED").
  • Registers the service in the manifest with foregroundServiceType="mediaPlayback" and an androidx.media3.session.MediaSessionService intent-filter.

Files to modify

src/main/java/org/thoughtcrime/securesms/WebxdcActivity.java

  1. Detect WebView audio state using WebChromeClient.onCreateWindow / overriding WebChromeClient with audio focus detection. The cleanest approach for WebView audio detection is:

    • Add a @JavascriptInterface method notifyAudioStarted(String title) and notifyAudioStopped() to InternalJSApi.
    • Inject a small script into webxdc.js or via evaluateJavascript on page load that attaches play/pause/ended event listeners to all <audio> and <video> elements (including dynamically created ones via a MutationObserver), and calls InternalJSApi.notifyAudioStarted(document.title) / notifyAudioStopped().
    • Inject this script after page load in WebChromeClient.onPageFinished equivalent — use webView.addJavascriptInterface already done, and call evaluateJavascript from WebViewClient.onPageFinished (override in WebxdcActivity's WebChromeClient setup).
  2. Connect to WebxdcMediaSessionService using MediaController (like ConversationActivity does with AudioPlaybackService):

    • In onCreate: build a MediaController against WebxdcMediaSessionService's SessionToken.
    • In onDestroy: release the MediaController and send WEBXDC_AUDIO_STOPPED.
  3. Send commands from JS callbacks:

    • notifyAudioStarted(title): sen...

This pull request was created from Copilot chat.

Copilot AI and others added 2 commits April 12, 2026 19:59
- Create WebxdcMediaSessionService: a MediaSessionService backed by a
  stub SimpleBasePlayer that posts the system media notification when
  webxdc apps play audio. Handles WEBXDC_AUDIO_STARTED/STOPPED/PAUSED/
  RESUMED custom SessionCommands from WebxdcActivity and relays
  notification play/pause button presses back to the WebView via
  local broadcasts.

- Modify WebxdcActivity:
  - Connect a MediaController to WebxdcMediaSessionService in onCreate
  - Add notifyAudioStarted/Stopped/Paused/Resumed JS interface methods
    in InternalJSApi (called from webxdc.js audio event listeners)
  - Register a BroadcastReceiver for notification pause/resume to call
    evaluateJavascript to pause/resume <audio>/<video> elements
  - Override pauseWebViewOnPause() to keep WebView running in background
    while audio is playing
  - Release controller and send AUDIO_STOPPED on destroy

- Modify WebViewActivity: add pauseWebViewOnPause() hook so subclasses
  can prevent webView.onPause() (e.g., during audio playback)

- Add audio/video event listener setup to webxdc.js that polls for
  audio/video elements and calls InternalJSApi.notifyAudio* when
  play/pause/ended events fire

- Register WebxdcMediaSessionService in AndroidManifest.xml with
  foregroundServiceType=mediaPlayback

Agent-Logs-Url: https://github.com/ArcaneChat/android/sessions/57af82f8-b482-4d5a-977f-fc74ffd48db3

Co-authored-by: adbenitez <24558636+adbenitez@users.noreply.github.com>
…e, simplify JS checks

Agent-Logs-Url: https://github.com/ArcaneChat/android/sessions/57af82f8-b482-4d5a-977f-fc74ffd48db3

Co-authored-by: adbenitez <24558636+adbenitez@users.noreply.github.com>
Copilot AI changed the title [WIP] Integrate Webxdc WebView audio playback with Android media session Integrate webxdc WebView audio playback with Android media session Apr 12, 2026
Copilot AI requested a review from adbenitez April 12, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants