feat: major improvements to native Telegram integration UX and streaming#1676
Open
hash-anmol wants to merge 9 commits into
Open
feat: major improvements to native Telegram integration UX and streaming#1676hash-anmol wants to merge 9 commits into
hash-anmol wants to merge 9 commits into
Conversation
This reverts commit c51c235.
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR expands the Telegram integration with inline command pickers, response/tool streaming (via editable messages), heartbeat updates, richer command registry, and better media delivery support; it also adds auto-queueing of messages received while a run is active (Telegram + WhatsApp), plus new tests around these behaviors.
Changes:
- Add Telegram streaming + heartbeat infrastructure (
draft_stream.py,heartbeat.py) and wire via extensions/hooks. - Introduce a shared integration command registry (
integration_commands.py) and Telegram inline keyboard command UI (command_ui.py), plus native Telegram command menu registration. - Extend Telegram media sending helpers (voice/audio/video) and update reply routing + add tests for sessions, streaming, heartbeat, media delivery, and error UI.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_telegram_sessions_picker.py | Adds coverage for Telegram session picker paging + session remapping. |
| tests/test_telegram_media_delivery.py | Verifies native media helper routing (voice/audio/video) and reply attachment routing. |
| tests/test_telegram_intermediate_response.py | Tests intermediate streaming behavior and tool progress grouping. |
| tests/test_telegram_heartbeat.py | Validates heartbeat formatting and start/stop behavior. |
| tests/test_telegram_error_ui.py | Covers friendly error classification and exception hook “send once + cleanup”. |
| plugins/_whatsapp_integration/helpers/handler.py | Queues messages and pauses typing when a WhatsApp run is already active. |
| plugins/_telegram_integration/helpers/telegram_client.py | Adds raw Bot API calls (send/edit), media helpers, and file-type detectors. |
| plugins/_telegram_integration/helpers/heartbeat.py | New Telegram heartbeat loop and progress “reason” formatting. |
| plugins/_telegram_integration/helpers/handler.py | Integrates command UI + unknown-command replies, queues during active run, routes more media types, and stream finalization. |
| plugins/_telegram_integration/helpers/error_ui.py | New friendly error formatting + classification. |
| plugins/_telegram_integration/helpers/draft_stream.py | New editable-message streaming for tool progress and response previews/finalization. |
| plugins/_telegram_integration/helpers/constants.py | Adds Telegram streaming/progress/heartbeat context keys. |
| plugins/_telegram_integration/helpers/command_ui.py | Adds inline keyboard pickers (model/project/agent/session) and stream/tools toggles + callback handling. |
| plugins/_telegram_integration/helpers/bot_manager.py | Registers Telegram command handlers from the shared command registry + sets Telegram command menu. |
| plugins/_telegram_integration/extensions/python/tool_execute_before/_45_telegram_draft_tool.py | Streams “tool start” events into Telegram progress messages. |
| plugins/_telegram_integration/extensions/python/tool_execute_after/_50_telegram_response.py | Streams intermediate response/tool completion, and falls back to send_telegram_reply when needed. |
| plugins/_telegram_integration/extensions/python/system_prompt/_20_telegram_context.py | Adds a system prompt segment describing Telegram command handling + lists commands. |
| plugins/_telegram_integration/extensions/python/response_stream/_45_telegram_draft_response.py | Streams response previews as the “response” tool emits data. |
| plugins/_telegram_integration/extensions/python/process_chain_end/_55_telegram_reply.py | Ensures heartbeat/stream state is cleaned up at end of processing. |
| plugins/_telegram_integration/extensions/python/message_loop_start/_45_telegram_draft_start.py | Starts streaming + heartbeat at message loop start. |
| plugins/_telegram_integration/extensions/python/job_loop/_10_telegram_bot.py | Removes /clear handler wiring and registers native Telegram command menu. |
| plugins/_telegram_integration/extensions/python/_functions/agent/Agent/handle_exception/end/_85_telegram_error.py | Sends a friendly Telegram error once and cleans up stream/heartbeat/typing state. |
| plugins/_telegram_integration/README.md | Updates docs describing new commands, streaming, queueing, and new helper modules/extensions. |
| helpers/integration_commands.py | Introduces command registry + help/status/agent/model/stream/tools/etc. command handlers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+326
to
+328
| if response.status != 200 or not data.get("ok"): | ||
| PrintStyle.debug(f"Telegram {method} failed: {data}") | ||
| return data if isinstance(data, dict) else {} |
Comment on lines
+323
to
+324
| async with aiohttp.ClientSession(timeout=timeout) as session: | ||
| async with session.post(url, json=payload) as response: |
|
|
||
|
|
||
| class TelegramFriendlyError(Extension): | ||
| async def execute(self, data: dict = {}, **kwargs): |
|
|
||
| async def execute( | ||
| self, | ||
| loop_data: LoopData = LoopData(), |
|
|
||
| class TelegramDraftStart(Extension): | ||
|
|
||
| async def execute(self, loop_data: LoopData = LoopData(), **kwargs): |
Comment on lines
406
to
+409
| context.set_data("chat_model_override", {"preset_name": preset_name}) | ||
| save_tmp_chat(context) | ||
| mark_dirty_for_context(context.id, reason="integration_commands.config_set") | ||
| return f"Switched config to {preset_name}." | ||
| return f"Switched model preset to {preset_name}." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a major quality-of-life and functional overhaul to Agent Zero's native Telegram integration plugin. It refines how streams, errors, and media are handled, making the Telegram interaction feel much more natively responsive.
Key Features & Functionalities Added
Telegram Session Picker: Added test_telegram_sessions_picker.py and backend logic to allow users to switch between Agent Zero contexts directly from the Telegram UI.
Native Media Attachments: Media generated or retrieved by the agent is now parsed and delivered as native Telegram media objects rather than raw URLs/text (test_telegram_media_delivery.py).
Long-Run Status Updates (Heartbeat): Added a heartbeat mechanism (test_telegram_heartbeat.py) that pushes periodic status updates to Telegram when the agent is engaged in long-running tasks, preventing the bot from appearing unresponsive.
Friendly Error Handling: Intercepts generic exception dumps and formats them into clean, readable Telegram messages (test_telegram_error_ui.py, _85_telegram_error.py).
Refined Streaming UX:
Intermediate Responses: Extracted intermediate agent thoughts into separate streams (test_telegram_intermediate_response.py).
Tool Streams Grouping: Tool execution streams are now properly grouped with their parent responses, cleaning up the message timeline.
Test Coverage
Added comprehensive test coverage for all new functionality. Specifically, 5 new test suites were added to cover the new behaviors:
test_telegram_intermediate_response.py
test_telegram_media_delivery.py
test_telegram_error_ui.py
test_telegram_heartbeat.py
test_telegram_sessions_picker.py
All tests pass locally, ensuring the new streaming logic does not break existing integrations.
Notes
Speech Mode Revert: A previous commit in this branch's history experimented with a Telegram Speech Mode. That functionality has been explicitly reverted (013a374) in this PR to keep the scope focused purely on text/media UX improvements.