Skip to content

Use imageGenerate for image modification requests #119

@sentry-junior

Description

@sentry-junior

Summary

When a user asks to modify or recreate an image (e.g. update a chart, change a visualization), the assistant currently skips imageGenerate and falls back to non-visual responses. It should recognize these as valid use cases for the image generation tool and attempt to fulfill them visually.

Background

A user requested an image modification (changing a chart). The assistant declined to use imageGenerate even though the task was plausibly in scope. The existing tool-usage guidance says to use imageGenerate when the user wants to visually represent something, but the assistant doesn't connect "modify this image" requests to that definition.

Scope

  • Update tool-usage instructions or skill guidance so the assistant maps image-modification intent (editing, recreating, updating a visual) to imageGenerate calls
  • Likely includes chart regeneration, diagram edits, and similar re-render tasks
  • Out of scope: pixel-level editing of existing rasters (not supported by the underlying tool)

Implementation

  • Clarify in tool-usage docs that "modify an image" and "update this chart/diagram" requests should trigger imageGenerate with a reconstructed prompt
  • Add guidance on how to infer prompt content from the existing image description + requested changes
  • Note limitations (generative output, not pixel-accurate edits)

Action taken on behalf of David Cramer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions