Summary
When a user asks to modify or recreate an image (e.g. update a chart, change a visualization), the assistant currently skips imageGenerate and falls back to non-visual responses. It should recognize these as valid use cases for the image generation tool and attempt to fulfill them visually.
Background
A user requested an image modification (changing a chart). The assistant declined to use imageGenerate even though the task was plausibly in scope. The existing tool-usage guidance says to use imageGenerate when the user wants to visually represent something, but the assistant doesn't connect "modify this image" requests to that definition.
Scope
- Update tool-usage instructions or skill guidance so the assistant maps image-modification intent (editing, recreating, updating a visual) to
imageGenerate calls
- Likely includes chart regeneration, diagram edits, and similar re-render tasks
- Out of scope: pixel-level editing of existing rasters (not supported by the underlying tool)
Implementation
- Clarify in tool-usage docs that "modify an image" and "update this chart/diagram" requests should trigger
imageGenerate with a reconstructed prompt
- Add guidance on how to infer prompt content from the existing image description + requested changes
- Note limitations (generative output, not pixel-accurate edits)
Action taken on behalf of David Cramer.
Summary
When a user asks to modify or recreate an image (e.g. update a chart, change a visualization), the assistant currently skips
imageGenerateand falls back to non-visual responses. It should recognize these as valid use cases for the image generation tool and attempt to fulfill them visually.Background
A user requested an image modification (changing a chart). The assistant declined to use
imageGenerateeven though the task was plausibly in scope. The existing tool-usage guidance says to useimageGeneratewhen the user wants to visually represent something, but the assistant doesn't connect "modify this image" requests to that definition.Scope
imageGeneratecallsImplementation
imageGeneratewith a reconstructed promptAction taken on behalf of David Cramer.