Support Image results in FunctionExecutionResult #7131
xukefaker
started this conversation in
Feature suggestions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am developing an LLM-based agent designed for paper analysis. The agent's core workflow is as follows:
The agent receives a markdown document (representing the paper's sections) as input.
The LLM may invoke the read_paper_images() tool to perform a detailed analysis of the paper's figures.
The read_paper_images() tool then returns the image data to the LLM."
The result of the read_paper_images() tool is:
However, I've observed that the$\text{FunctionExecutionResult}$ object flattens this dictionary into a single string (specifically, $\text{FunctionExecutionResult.content}$ ). Given this, how can the LLM effectively read and process the images returned by the function/tool?
This is also related to #5250
Beta Was this translation helpful? Give feedback.
All reactions