[WIP]refactor!: redesign pipeline with chain-based engine and NodeStar system#4960
Draft
Raven95676 wants to merge 11 commits intoAstrBotDevs:masterfrom
Draft
[WIP]refactor!: redesign pipeline with chain-based engine and NodeStar system#4960Raven95676 wants to merge 11 commits intoAstrBotDevs:masterfrom
Raven95676 wants to merge 11 commits intoAstrBotDevs:masterfrom
Conversation
…itState validation, remove NodeResult.SKIP
Contributor
There was a problem hiding this comment.
Pull request overview
This WIP PR introduces a new chain-based pipeline execution architecture (NodeStar + ChainConfig/Router/Executor/WaitRegistry) to replace the legacy multi-stage “onion” pipeline, and updates the dashboard to manage “Chain Routing” instead of session-based rule management.
Changes:
- Replace legacy pipeline stages/scheduler with a chain-based execution engine (routing, node contexts/packets, wait/resume, runtime feature flags).
- Introduce NodeStar plugin model + node-scoped config schemas and several builtin pipeline nodes (agent/stt/tts/t2i/knowledge_base/file_extract/content_safety).
- Update WebUI routes/navigation/i18n and some dashboard APIs to align with chain/config_id routing.
Reviewed changes
Copilot reviewed 144 out of 145 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| dashboard/src/router/MainRoutes.ts | Rename route to ChainManagement and update path/component. |
| dashboard/src/layouts/full/vertical-sidebar/sidebarItem.ts | Update sidebar entry to Chain Routing. |
| dashboard/src/i18n/translations.ts | Swap session-management module for chain-management module. |
| dashboard/src/i18n/locales/zh-CN/features/session-management.json | Remove legacy session management locale. |
| dashboard/src/i18n/locales/zh-CN/features/config-metadata.json | Remove legacy STT/TTS/file_extract metadata keys. |
| dashboard/src/i18n/locales/zh-CN/features/chain-management.json | Add chain-management locale strings (ZH). |
| dashboard/src/i18n/locales/zh-CN/core/navigation.json | Rename nav key to chainManagement (ZH). |
| dashboard/src/i18n/locales/en-US/features/session-management.json | Remove legacy session management locale. |
| dashboard/src/i18n/locales/en-US/features/config-metadata.json | Remove legacy STT/TTS/file_extract metadata keys. |
| dashboard/src/i18n/locales/en-US/features/chain-management.json | Add chain-management locale strings (EN). |
| dashboard/src/i18n/locales/en-US/core/navigation.json | Rename nav key to chainManagement (EN). |
| dashboard/src/i18n/loader.ts | Load chain-management module instead of session-management. |
| dashboard/src/components/shared/ObjectEditor.vue | Add “lock template keys” behavior and conditional custom key editing. |
| dashboard/src/components/chat/StandaloneChat.vue | Remove config-id prop wiring to ChatInput. |
| dashboard/src/components/chat/ChatInput.vue | Remove ConfigSelector and related props/computed logic. |
| dashboard/src/components/RuleEditor.vue | Add rule editor UI for chain match rules. |
| astrbot/dashboard/server.py | Wire up ChainManagementRoute (via routes import). |
| astrbot/dashboard/routes/t2i.py | Reload pipeline executors after template changes. |
| astrbot/dashboard/routes/cron.py | Include config_id in cron job payload. |
| astrbot/dashboard/routes/config.py | Shift API naming/behavior toward config_id and remove UMOP routing endpoints. |
| astrbot/dashboard/routes/chat.py | Remove UMOP config-route cleanup on webchat session delete. |
| astrbot/dashboard/routes/init.py | Export ChainManagementRoute; remove SessionManagementRoute export. |
| astrbot/core/utils/session_waiter.py | Disallow SessionWaiter usage in NodeStar/chain context. |
| astrbot/core/utils/migra_helper.py | Add migration hook for chain configs and refactor UMOP router creation. |
| astrbot/core/umop_config_router.py | Rename mapping to umop_to_config_id and update related APIs. |
| astrbot/core/tools/cron_tools.py | Add config_id to tool-originated cron payload. |
| astrbot/core/star/star_manager.py | Load optional node config schema for NodeStar plugins. |
| astrbot/core/star/star_base.py | New shared Star base class extracted from star/init.py. |
| astrbot/core/star/star.py | Add node_schema field to StarMetadata and adjust imports. |
| astrbot/core/star/session_plugin_manager.py | Remove legacy per-session plugin enable/disable manager. |
| astrbot/core/star/session_llm_manager.py | Remove legacy per-session service enable/disable manager. |
| astrbot/core/star/register/star.py | Fix import path for StarMetadata/star_map after refactor. |
| astrbot/core/star/node_star.py | Introduce NodeStar base class + NodeResult + streaming collector helper. |
| astrbot/core/star/modality.py | Add modality extraction utilities for routing. |
| astrbot/core/star/init.py | Re-export Star/NodeStar/Modality APIs and clean up module structure. |
| astrbot/core/provider/manager.py | Deprecate legacy provider switching APIs and adjust STT/TTS resolution. |
| astrbot/core/platform/sources/webchat/webchat_adapter.py | Only set enable_streaming extra if explicitly provided by client payload. |
| astrbot/core/pipeline/whitelist_check/stage.py | Remove legacy stage implementation (replaced by AccessController). |
| astrbot/core/pipeline/waking_check/stage.py | Remove legacy waking stage implementation. |
| astrbot/core/pipeline/system/star_yield.py | Add compatibility layer for legacy Star async-generator (yield) handlers. |
| astrbot/core/pipeline/system/session_utils.py | Extract unique-session-id builder helpers. |
| astrbot/core/pipeline/system/rate_limit.py | Add chain-compatible fixed-window rate limiter. |
| astrbot/core/pipeline/system/event_preprocessor.py | Add system-level preprocessing (admins/unique session/path mapping). |
| astrbot/core/pipeline/system/access_control.py | Add chain-compatible whitelist check. |
| astrbot/core/pipeline/system/init.py | Export new system pipeline utilities. |
| astrbot/core/pipeline/stage.py | Remove legacy Stage abstraction. |
| astrbot/core/pipeline/session_status_check/stage.py | Remove legacy session status stage. |
| astrbot/core/pipeline/scheduler.py | Remove legacy PipelineScheduler. |
| astrbot/core/pipeline/rate_limit_check/stage.py | Remove legacy rate limit stage. |
| astrbot/core/pipeline/process_stage/stage.py | Remove legacy processing stage. |
| astrbot/core/pipeline/process_stage/method/star_request.py | Remove legacy star request sub-stage. |
| astrbot/core/pipeline/process_stage/method/agent_request.py | Remove legacy agent request sub-stage. |
| astrbot/core/pipeline/preprocess_stage/stage.py | Remove legacy preprocess stage (STT/path mapping/ack emoji, etc.). |
| astrbot/core/pipeline/engine/wait_registry.py | Add wait/resume registry for multi-turn node WAIT behavior. |
| astrbot/core/pipeline/engine/rule_matcher.py | Add match rule evaluation engine (umo/modality/text regex). |
| astrbot/core/pipeline/engine/router.py | Add ChainRouter loading configs from DB and routing events to chains. |
| astrbot/core/pipeline/engine/node_context.py | Add node context stack + NodePacket model. |
| astrbot/core/pipeline/engine/chain_runtime_flags.py | Add per-chain runtime feature toggles (llm/stt/tts/t2i). |
| astrbot/core/pipeline/engine/chain_executor.py | Add sequential chain executor with node init, outputs, and WAIT support. |
| astrbot/core/pipeline/engine/chain_config.py | Add DB model + runtime ChainConfig and node normalization/UUID strategy. |
| astrbot/core/pipeline/context_utils.py | Remove legacy call_handler implementation, keep hook calls. |
| astrbot/core/pipeline/context.py | Adjust PipelineContext to new executor model and typing exports. |
| astrbot/core/pipeline/content_safety_check/strategies/strategy.py | Remove legacy content safety strategy selector (moved to builtin node). |
| astrbot/core/pipeline/content_safety_check/strategies/keywords.py | Remove legacy keywords strategy. |
| astrbot/core/pipeline/content_safety_check/strategies/baidu_aip.py | Remove legacy Baidu AIP strategy. |
| astrbot/core/pipeline/content_safety_check/strategies/init.py | Remove legacy strategy base class. |
| astrbot/core/pipeline/content_safety_check/stage.py | Remove legacy content safety stage. |
| astrbot/core/pipeline/agent/types.py | Add AgentRunOutcome for new agent execution flow. |
| astrbot/core/pipeline/agent/third_party.py | Refactor third-party agent runner into executor returning AgentRunOutcome. |
| astrbot/core/pipeline/agent/runner_config.py | Add node-config-based agent runner resolution helpers. |
| astrbot/core/pipeline/agent/executor.py | Add unified AgentExecutor (internal + third-party) with chain runtime flags. |
| astrbot/core/pipeline/agent/init.py | Export AgentExecutor. |
| astrbot/core/pipeline/init.py | Remove legacy stage ordering exports. |
| astrbot/core/persona_mgr.py | Allow resolving default persona via config_id (chain-routed config). |
| astrbot/core/message/message_event_result.py | Add collect_streaming_result helper. |
| astrbot/core/event_bus.py | Route events via ChainRouter + resume via WaitRegistry; dispatch PipelineExecutor. |
| astrbot/core/db/migration/migra_45_to_46.py | Update migration naming and UMOP mapping keys (config_id). |
| astrbot/core/cron/manager.py | Resolve config by config_id from cron payload for permissions. |
| astrbot/core/core_lifecycle.py | Replace scheduler with PipelineExecutor mapping + initialize ChainRouter. |
| astrbot/core/config/node_config.py | Add persisted node-scoped config (AstrBotNodeConfig) keyed by node/chain/uuid. |
| astrbot/core/config/init.py | Export AstrBotNodeConfig. |
| astrbot/core/computer/tools/shell.py | Pass config_id into sandbox booter resolution. |
| astrbot/core/computer/tools/python.py | Pass config_id into sandbox booter resolution. |
| astrbot/core/computer/tools/fs.py | Pass config_id into sandbox booter resolution. |
| astrbot/core/computer/computer_client.py | Resolve sandbox config via config_id. |
| astrbot/core/astr_main_agent_resources.py | Resolve config via chain config_id and remove session-scoped KB config. |
| astrbot/core/astr_agent_tool_exec.py | Resolve provider id with event-aware selection. |
| astrbot/builtin_stars/web_searcher/main.py | Resolve config via chain config_id; adjust Baidu MCP init signature. |
| astrbot/builtin_stars/tts/metadata.yaml | Add builtin TTS node metadata. |
| astrbot/builtin_stars/tts/main.py | Add TTS node implementation (chain runtime flag + provider resolution). |
| astrbot/builtin_stars/tts/_node_config_schema.json | Add TTS node config schema. |
| astrbot/builtin_stars/t2i/metadata.yaml | Add builtin T2I node metadata. |
| astrbot/builtin_stars/t2i/main.py | Add T2I node implementation (chain runtime flag + template selection). |
| astrbot/builtin_stars/t2i/_node_config_schema.json | Add T2I node config schema. |
| astrbot/builtin_stars/stt/metadata.yaml | Add builtin STT node metadata. |
| astrbot/builtin_stars/stt/main.py | Add STT node implementation (chain runtime flag + provider resolution). |
| astrbot/builtin_stars/stt/_node_config_schema.json | Add STT node config schema. |
| astrbot/builtin_stars/session_controller/main.py | Resolve config via chain config_id for behavior. |
| astrbot/builtin_stars/knowledge_base/metadata.yaml | Add knowledge-base retrieval node metadata. |
| astrbot/builtin_stars/knowledge_base/main.py | Add knowledge-base retrieval node implementation. |
| astrbot/builtin_stars/knowledge_base/_node_config_schema.json | Add KB node config schema. |
| astrbot/builtin_stars/file_extract/metadata.yaml | Add file extraction node metadata. |
| astrbot/builtin_stars/file_extract/main.py | Add file extraction node implementation. |
| astrbot/builtin_stars/file_extract/_node_config_schema.json | Add file extraction node config schema. |
| astrbot/builtin_stars/content_safety/strategies.py | Add content safety strategies for node-based checks. |
| astrbot/builtin_stars/content_safety/requirements.txt | Add plugin-level dependency list (baidu-aip). |
| astrbot/builtin_stars/content_safety/metadata.yaml | Add content safety node metadata. |
| astrbot/builtin_stars/content_safety/main.py | Add content safety node implementation (input/output checks). |
| astrbot/builtin_stars/content_safety/_node_config_schema.json | Add content safety node config schema. |
| astrbot/builtin_stars/builtin_commands/main.py | Add STT command and reorder built-in commands. |
| astrbot/builtin_stars/builtin_commands/commands/tts.py | Change TTS toggle to per-chain runtime flag. |
| astrbot/builtin_stars/builtin_commands/commands/t2i.py | Change T2I toggle to per-chain runtime flag. |
| astrbot/builtin_stars/builtin_commands/commands/stt.py | Add STT toggle command (per-chain runtime flag). |
| astrbot/builtin_stars/builtin_commands/commands/llm.py | Change LLM toggle to per-chain runtime flag. |
| astrbot/builtin_stars/builtin_commands/commands/conversation.py | Make conversation commands chain/node-aware for runner/provider/persona. |
| astrbot/builtin_stars/builtin_commands/commands/admin.py | Resolve config via chain config_id for whitelist ops. |
| astrbot/builtin_stars/builtin_commands/commands/_node_binding.py | Add helpers to locate chain nodes and node configs. |
| astrbot/builtin_stars/builtin_commands/commands/init.py | Export STTCommand. |
| astrbot/builtin_stars/astrbot/main.py | Resolve config via chain config_id; adjust active reply provider selection. |
| astrbot/builtin_stars/astrbot/long_term_memory.py | Make provider selection event-aware for image captioning. |
| astrbot/builtin_stars/agent/metadata.yaml | Add builtin agent node metadata. |
| astrbot/builtin_stars/agent/main.py | Add agent node implementation using AgentExecutor. |
| astrbot/builtin_stars/agent/_node_config_schema.json | Add agent node config schema (runner + provider/persona overrides). |
| astrbot/api/star/init.py | Export NodeStar/NodeResult in public API. |
Comments suppressed due to low confidence (3)
astrbot/builtin_stars/astrbot/main.py:12
- This class does not call Star.init during initialization. (Main.init may be missing a call to a base class init)
class Main(star.Star):
astrbot/builtin_stars/web_searcher/main.py:20
- This class does not call Star.init during initialization. (Main.init may be missing a call to a base class init)
astrbot/builtin_stars/builtin_commands/main.py:21 - This class does not call Star.init during initialization. (Main.init may be missing a call to a base class init)
class Main(star.Star):
Comment on lines
+292
to
+299
| if (hasTemplateSchema.value && !allowCustomKeys.value) { | ||
| for (const [key, template] of Object.entries(templateSchema.value)) { | ||
| const rawValue = props.modelValue[key] | ||
| let _type = template.type || (typeof rawValue === 'object' ? 'json' : typeof rawValue) | ||
| let _value = _type === 'json' ? JSON.stringify(rawValue) : rawValue | ||
| if (_value === undefined || _value === null) { | ||
| _value = template.default !== undefined ? template.default : getDefaultValueForType(_type) | ||
| } |
There was a problem hiding this comment.
When template keys are locked, null values get stringified to the literal string "null" (because typeof null === 'object'), so the default value fallback won't run. This can persist invalid values and break type handling. Consider checking rawValue for null/undefined before inferring type / JSON.stringify, and fall back to template.default (or getDefaultValueForType) when rawValue is null.
Comment on lines
+446
to
+451
| config_id = self.acm.create_conf(name=name, config=config) | ||
| return ( | ||
| Response() | ||
| .ok(message="创建成功", data={"config_id": config_id}) | ||
| .__dict__ | ||
| ) |
There was a problem hiding this comment.
This endpoint changes the response payload key from "conf_id" to "config_id". Since other endpoints still accept legacy request keys ("id"), consider returning both keys (or keeping "conf_id" as an alias) to avoid breaking existing dashboard/client code that expects "conf_id".
Comment on lines
836
to
+850
| async def post_astrbot_configs(self): | ||
| data = await request.json | ||
| config = data.get("config", None) | ||
| conf_id = data.get("conf_id", None) | ||
| config_id = data.get("config_id") | ||
|
|
||
| try: | ||
| # 不更新 provider_sources, provider, platform | ||
| # 这些配置有单独的接口进行更新 | ||
| if conf_id == "default": | ||
| if config_id == "default": | ||
| no_update_keys = ["provider_sources", "provider", "platform"] | ||
| for key in no_update_keys: | ||
| config[key] = self.acm.default_conf[key] | ||
|
|
||
| await self._save_astrbot_configs(config, conf_id) | ||
| await self.core_lifecycle.reload_pipeline_scheduler(conf_id) | ||
| await self._save_astrbot_configs(config, config_id) | ||
| await self.core_lifecycle.reload_pipeline_executor(config_id) |
There was a problem hiding this comment.
post_astrbot_configs now only reads "config_id" from the request body. Other endpoints in this file accept legacy keys ("conf_id"/"id"); without a fallback here, older clients will start failing with "配置文件 None 不存在". Consider accepting "conf_id" (and/or "id") as an alias, similar to delete_abconf/update_abconf.
| @@ -0,0 +1 @@ | |||
| baidu-aip No newline at end of file | |||
There was a problem hiding this comment.
This adds a new dependency (baidu-aip). The PR checklist indicates no new dependencies were introduced; please update the checklist/PR description accordingly and ensure the project’s dependency installation flow will actually install this plugin requirement in production deployments.
|
|
||
| async def _post_send(self): | ||
| """调度器会在执行 send() 后调用该方法 deprecated in v3.5.18""" | ||
| async def _post_send(self, message: MessageChain | None = None, **_): |
There was a problem hiding this comment.
Overridden method signature does not match call, where it is passed an argument named 'stream'. Overriding method method QQOfficialMessageEvent._post_send matches the call.
Overridden method signature does not match call, where it is passed an argument named 'stream'. Overriding method method QQOfficialMessageEvent._post_send matches the call.
| import re | ||
| from typing import Any | ||
|
|
||
| import astrbot.core.message.components as Comp |
There was a problem hiding this comment.
Module 'astrbot.core.message.components' is imported with both 'import' and 'import from'.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
impl #1948
Caution
这是一个破坏性变更
Need More and More Test
Modifications / 改动点
用chain-based线性流水线替代原有 9 阶段洋葱模型pipeline，作为面向workflow扩展的渐进式重构
新增 Chain 系统：
NodeStar：可挂载到 Chain 中的处理节点基类，
process()返回NodeResult（CONTINUE/STOP/WAIT/SKIP）ChainConfig：DB 持久化的 Chain 配置，定义节点顺序与匹配规则
ChainExecutor：顺序执行 Chain 节点，通过
NodeContext/NodePacket实现节点间数据传递WaitRegistry：多轮等待/恢复机制
This is NOT a breaking change. / 这不是一个破坏性变更。
Screenshots or Test Results / 运行截图或测试结果
Checklist / 检查清单
requirements.txt和pyproject.toml文件相应位置。/ I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations inrequirements.txtandpyproject.toml.