Skip to content

feat: add a sandbox image-viewing tool for agent vision tasks #178

Description

@QueryPlanner

Summary

Add an agent-accessible tool for viewing image files stored in the sandbox.

The tool should let the agent provide a sandbox image path and receive the image through a model-compatible visual input channel. This gives the agent a reliable way to inspect images that arrived as Telegram files/documents or were restored from attachment storage.

Motivating use case

Send multiple images to the Black Key Telegram bot and ask:

create 2 collage - randomly select 6 pictures in a colloege

The agent should be able to inspect the individual images stored in the sandbox, choose the requested images, and create the collages.

Proposed behavior

  • Add a tool such as sandbox_view_image.
  • Accept a sandbox-relative image path or an attachment reference.
  • Validate that the target is an allowed sandbox file.
  • Support common image formats and MIME types.
  • Return the image in a model-compatible visual response, not only metadata or a text path.
  • Preserve individual image boundaries when viewing multiple images.
  • Return a clear, bounded error for missing, invalid, or unsupported files.

Security and reliability requirements

  • Restrict reads to the active sandbox and approved attachment locations.
  • Do not allow path traversal or arbitrary host filesystem access.
  • Enforce reasonable file-size and image-dimension limits.
  • Keep the tool read-only.
  • Ensure the tool works after attachment restoration or sandbox reconnect when the file is available.

Acceptance criteria

  • An agent can call the tool with an image stored in the sandbox and receive a visual representation it can inspect.
  • The tool supports images uploaded to Telegram as files/documents.
  • Multiple images can be viewed independently for selection and collage operations.
  • Missing, unsupported, oversized, and invalid images produce actionable errors.
  • Path validation prevents access outside the active sandbox.
  • Unit and integration tests cover single-image viewing, multiple-image viewing, restored attachments, and invalid paths.
  • The tool is documented and registered in the agent's available tool set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions