[DataProcessor] Move image_processor to unified directory and add MultiModalProcessor by luukunn · Pull Request #7109 · PaddlePaddle/FastDeploy

luukunn · 2026-03-31T08:34:43Z

Motivation

本 PR 旨在对多模态（VL）相关的 image processor 做统一目录管理与架构优化：

统一目录结构：将原先分散在各 VL 子目录（qwen_vl_processor/、qwen3_vl_processor/、paddleocr_vl_processor/、ernie4_5_vl_processor/）下的 image_processor / image_preprocessor 逐步迁移到统一的 fastdeploy/input/image_processors/ 目录下，降低代码碎片化程度，便于后续维护和扩展新模型的 image processor。
新增 MultiModalProcessor：引入 MultiModalProcessor 作为多模态场景下的统一入口，封装 VL 请求处理流程与模型类型分发逻辑，由 InputPreprocessor 统一调用，替代原先在 preprocess.py 中按模型类型分别创建各 VL processor 的冗余代码。
保留向后兼容：在各旧路径下保留兼容导入入口（shim），确保已有的外部引用不会因迁移而中断。

Modifications

新增文件（6 个）

文件路径	说明
`fastdeploy/input/multimodal_processor.py`	新增 `MultiModalProcessor` 统一多模态 processor，封装 VL 请求处理流程与模型类型分发
`fastdeploy/input/image_processors/adaptive_processor.py`	从 `ernie4_5_vl_processor/image_preprocessor/` 迁移 `AdaptiveImageProcessor` 实现
`fastdeploy/input/image_processors/qwen_processor.py`	从 `qwen_vl_processor/` 迁移 QwenVL `ImageProcessor` 实现
`fastdeploy/input/image_processors/qwen3_processor.py`	从 `qwen3_vl_processor/` 迁移 Qwen3VL `ImageProcessor` 实现
`fastdeploy/input/image_processors/paddleocr_processor.py`	从 `paddleocr_vl_processor/` 迁移 PaddleOCRVL `ImageProcessor` 实现
`tests/input/test_multimodal_processor.py`	新增 `MultiModalProcessor` 单元测试

兼容层（旧路径转发，4 处修改）

旧路径	转发到新路径
`fastdeploy/input/qwen_vl_processor/image_processor.py`	→ `image_processors.qwen_processor`
`fastdeploy/input/qwen3_vl_processor/image_processor.py`	→ `image_processors.qwen3_processor`
`fastdeploy/input/paddleocr_vl_processor/image_processor.py`	→ `image_processors.paddleocr_processor`
`fastdeploy/input/ernie4_5_vl_processor/image_preprocessor/*`	→ `image_processors.adaptive_processor`

其他修改（3 处）

fastdeploy/input/image_processors/__init__.py — 更新导出，汇总新增的 image processor 模块
fastdeploy/input/preprocess.py — 多模态分支统一改为创建 MultiModalProcessor，移除原先按模型类型分别创建各 VL processor 的冗余逻辑
tests/input/test_image_preprocessor_adaptive.py — 适配 patch 路径到新模块位置

Usage or Command

Accuracy Tests

Checklist

Add at least a tag in the PR title.
- Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
- You can add new tags based on the PR content, but the semantics must be clear.
Format your code, run pre-commit before commit.
Add unit tests. Please write the reason in this PR if no unit tests.
Provide accuracy results.
If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

paddle-bot · 2026-03-31T08:35:09Z

Thanks for your contribution!

Copilot

Pull request overview

该 PR 旨在将多模态（VL）相关的 image processor 与 processor 路由逻辑做统一/迁移：把原先分散在各 VL 子目录下的 image_processor / image_preprocessor 逐步迁移到 fastdeploy/input/image_processors/，并新增一个统一的 MultiModalProcessor 供 InputPreprocessor 在多模态场景下分发使用，同时保留旧路径的兼容导入入口。

Changes:

新增 fastdeploy/input/multimodal_processor.py，用 model_type 统一封装 QwenVL/Qwen3VL/PaddleOCRVL/Ernie4.5VL 的请求处理分发。
新增 fastdeploy/input/image_processors/ 下的多个 processor 实现，并将旧 VL 目录内的 image_processor 模块改为兼容层转发导入。
更新相关单测 patch 路径，使其指向新的 image_processors 模块位置。

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
tests/input/test_image_preprocessor_adaptive.py	更新 patch 目标路径以匹配 image preprocessor 迁移后的模块位置
fastdeploy/input/preprocess.py	多模态分支统一改为创建 `MultiModalProcessor`（替代原先按架构分别 import 的处理器）
fastdeploy/input/multimodal_processor.py	新增统一多模态 processor，封装 VL 请求处理流程与模型类型分发
fastdeploy/input/image_processors/init.py	汇总导出新的 image processors（便于统一 import）
fastdeploy/input/image_processors/adaptive_processor.py	新增/迁移 AdaptiveImageProcessor 实现（从原 Ernie4.5 VL 路径迁移）
fastdeploy/input/image_processors/qwen_processor.py	新增/迁移 QwenVL ImageProcessor 实现
fastdeploy/input/image_processors/qwen3_processor.py	新增/迁移 Qwen3VL ImageProcessor 实现
fastdeploy/input/image_processors/paddleocr_processor.py	新增/迁移 PaddleOCRVL ImageProcessor 实现
fastdeploy/input/qwen_vl_processor/image_processor.py	旧路径兼容层：转发导入到新 `image_processors.qwen_processor`
fastdeploy/input/qwen3_vl_processor/image_processor.py	旧路径兼容层：转发导入到新 `image_processors.qwen3_processor`
fastdeploy/input/paddleocr_vl_processor/image_processor.py	旧路径兼容层：转发导入到新 `image_processors.paddleocr_processor`
fastdeploy/input/ernie4_5_vl_processor/image_preprocessor/image_preprocessor_adaptive.py	旧路径兼容层：转发导入到新 `image_processors.adaptive_processor`
fastdeploy/input/ernie4_5_vl_processor/image_preprocessor/get_image_preprocessor.py	旧路径兼容层：转发导入到新 `image_processors.adaptive_processor`
fastdeploy/input/ernie4_5_vl_processor/image_preprocessor/init.py	旧路径兼容层：从新模块重导出 AdaptiveImageProcessor/get_image_preprocessor

fastdeploy/input/preprocess.py

fastdeploy/input/image_processors/adaptive_processor.py

fastdeploy/input/image_processors/paddleocr_processor.py

fastdeploy/input/multimodal_processor.py

fastdeploy/input/preprocess.py

fastdeploy/input/image_processors/adaptive_processor.py

codecov-commenter · 2026-03-31T10:29:50Z

Codecov Report

❌ Patch coverage is 90.58488% with 66 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@c29e86f). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
fastdeploy/input/multimodal_processor.py	93.79%	8 Missing and 8 partials ⚠️
...astdeploy/input/image_processors/qwen_processor.py	83.52%	6 Missing and 8 partials ⚠️
...ploy/input/image_processors/paddleocr_processor.py	86.17%	11 Missing and 2 partials ⚠️
...stdeploy/input/image_processors/qwen3_processor.py	84.70%	6 Missing and 7 partials ⚠️
...eploy/input/image_processors/adaptive_processor.py	96.29%	5 Missing and 1 partial ⚠️
fastdeploy/input/preprocess.py	57.14%	3 Missing ⚠️
...essor/image_preprocessor/get_image_preprocessor.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #7109   +/-   ##
==========================================
  Coverage           ?   73.81%           
==========================================
  Files              ?      382           
  Lines              ?    53112           
  Branches           ?     8302           
==========================================
  Hits               ?    39206           
  Misses             ?    11192           
  Partials           ?     2714

Flag	Coverage Δ
GPU	`73.81% <90.58%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

fastdeploy/input/image_processors/adaptive_processor.py

fastdeploy/input/preprocess.py

fastdeploy-bot

🤖 AI Code Review | 2026-04-01 18:44:15 CST

📋 Review 摘要

PR 概述：将分散的 VL 模型图像处理器统一到 fastdeploy/input/image_processors/ 目录，并新增统一的 MultiModalProcessor 类

变更范围：fastdeploy/input/ 下的图像处理和多模态处理模块

影响面 Tag：DataProcessor

📝 PR 规范检查

PR 描述中 Motivation 和 Modifications 部分未填写，建议补充以便 Reviewer 了解变更目的。

描述建议（可直接复制）：

## Motivation

将分散在各 VL 处理器目录下的图像处理代码统一到 `fastdeploy/input/image_processors/` 目录，便于维护和复用。同时新增 `MultiModalProcessor` 统一处理多模态输入，减少重复代码。

## Modifications

1. 新增 `fastdeploy/input/image_processors/` 目录，包含：
   - `adaptive_processor.py`: 自适应图像处理器（从 ernie4_5_vl 迁移）
   - `qwen_processor.py`: Qwen2.5-VL 图像处理器
   - `qwen3_processor.py`: Qwen3-VL 图像处理器
   - `paddleocr_processor.py`: PaddleOCR-VL 图像处理器

2. 新增 `fastdeploy/input/multimodal_processor.py`: 统一多模态处理器

3. 原有路径保留向后兼容性导入

4. 更新 `preprocess.py` 使用新的 `MultiModalProcessor`

问题

级别	文件	概述
🔴 Bug	`qwen_processor.py:195`	循环内修改 `do_rescale` 和归一化参数导致多图处理异常
🔴 Bug	`qwen3_processor.py:195`	同上问题

总体评价

代码重构思路清晰，向后兼容性做得很好。但 qwen_processor.py 和 qwen3_processor.py 中存在循环内变量修改的 Bug，会导致批量图像处理时只有第一张图片被正确处理。建议修复后合并。

fastdeploy/input/image_processors/qwen_processor.py

fastdeploy/input/image_processors/qwen3_processor.py

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.

fastdeploy/input/multimodal_processor.py

fastdeploy/input/image_processors/qwen_processor.py

fastdeploy/input/image_processors/qwen3_processor.py

fastdeploy/input/image_processors/adaptive_processor.py

tests/input/test_multimodal_processor.py

fastdeploy/input/image_processors/paddleocr_processor.py

fastdeploy-bot

🤖 AI Code Review

📋 Review 摘要

PR 概述：将 VL 相关 image processor 迁移到统一目录 fastdeploy/input/image_processors/，并新增 MultiModalProcessor 作为多模态处理的统一入口。
变更范围：fastdeploy/input/ 目录下的 processor 模块
影响面 Tag：[DataProcessor]

问题

未发现阻塞性问题。

总体评价

这是一次良好的代码重构 PR：

目录结构优化：将分散的 image processor 统一到 image_processors/ 目录，降低了代码碎片化
架构改进：新增 MultiModalProcessor 统一了四种 VL 模型的处理逻辑入口，preprocess.py 的代码更加简洁
向后兼容：保留了旧路径的兼容导入（shim），不会破坏已有的外部引用
测试覆盖：新增了 test_multimodal_processor.py 单元测试

代码实现规范，兼容性考虑充分。

luukunn added 4 commits March 30, 2026 18:36

first commit

e91968d

step 9~10

a48fc5f

update multimodal

f5bdd67

update multimodal

7646a49

Copilot AI review requested due to automatic review settings March 31, 2026 08:34

luukunn had a problem deploying to Metax_ci March 31, 2026 08:34 — with GitHub Actions Failure

Copilot started reviewing on behalf of luukunn March 31, 2026 08:35 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

fix load tokenizer

d9199ab

luukunn had a problem deploying to Metax_ci April 1, 2026 07:13 — with GitHub Actions Failure

Merge branch 'develop' into merge_processor_3

890013c

Copilot AI review requested due to automatic review settings April 1, 2026 07:41

luukunn temporarily deployed to Metax_ci April 1, 2026 07:41 — with GitHub Actions Inactive

Copilot started reviewing on behalf of luukunn April 1, 2026 07:42 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

fastdeploy/input/image_processors/adaptive_processor.py Outdated Show resolved Hide resolved

fastdeploy/input/image_processors/adaptive_processor.py Show resolved Hide resolved

fastdeploy/input/preprocess.py Show resolved Hide resolved

fastdeploy/input/preprocess.py Show resolved Hide resolved

add unit test

70c5af3

luukunn temporarily deployed to Metax_ci April 1, 2026 10:02 — with GitHub Actions Inactive

luukunn changed the title ~~Merge processor~~ [Optimization]Move image_processor Apr 1, 2026

luukunn changed the title ~~[Optimization]Move image_processor~~ [Optimization] Move image_processor to unified directory and add MultiModalProcessor Apr 1, 2026

fastdeploy-bot suggested changes Apr 1, 2026

View reviewed changes

fastdeploy/input/image_processors/qwen_processor.py Show resolved Hide resolved

fastdeploy/input/image_processors/qwen3_processor.py Show resolved Hide resolved

luukunn requested review from LiqinruiG and jiangjiajun April 1, 2026 11:07

fix unit test & AdaptiveImageProcessor

d1663a3

Copilot AI review requested due to automatic review settings April 1, 2026 11:41

luukunn temporarily deployed to Metax_ci April 1, 2026 11:41 — with GitHub Actions Inactive

Copilot started reviewing on behalf of luukunn April 1, 2026 11:42 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

Delete unused code

47658fe

luukunn temporarily deployed to Metax_ci April 1, 2026 13:23 — with GitHub Actions Inactive

fastdeploy-bot approved these changes Apr 1, 2026

View reviewed changes

luukunn changed the title ~~[Optimization] Move image_processor to unified directory and add MultiModalProcessor~~ [DataProcessor] Move image_processor to unified directory and add MultiModalProcessor Apr 2, 2026

Conversation

luukunn commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

新增文件（6 个）

兼容层（旧路径转发，4 处修改）

其他修改（3 处）

Usage or Command

Accuracy Tests

Checklist

Uh oh!

paddle-bot bot commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fastdeploy-bot left a comment

Choose a reason for hiding this comment

📋 Review 摘要

📝 PR 规范检查

问题

总体评价

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fastdeploy-bot left a comment

Choose a reason for hiding this comment

📋 Review 摘要

问题

总体评价

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

luukunn commented Mar 31, 2026 •

edited

Loading

codecov-commenter commented Mar 31, 2026 •

edited

Loading