feat: Add tool result offloading

[sjrl]

Related Issues

• fixes https://github.com/deepset-ai/haystack-private/issues/405

Proposed Changes:

Added tool result offloading for the Agent via the new haystack.hooks.tool_result_offloading module.
ToolResultOffloadHook is an after_tool hook that writes selected tool results to a ToolResultStore and
replaces them in the conversation with a compact pointer (reference plus a short preview), so the next LLM call
sees a reference instead of the full result. This helps manage the context window and is a step towards letting an
Agent operate on offloaded results with follow-up tools (e.g. a bash tool reading the referenced files).

Configure what gets offloaded, per tool, via offload_strategies:

• Map a tool name, a tuple of tool names, or the wildcard "*" to an OffloadPolicy. More specific keys win over
  "*"; a tool with no matching key is not offloaded.
• Built-in policies: AlwaysOffload, NeverOffload, and OffloadOverChars(threshold) (offload when the result
  exceeds a character threshold).
• For custom conditions, implement the OffloadPolicy protocol (it provides default to_dict/from_dict, so a
  stateless policy only needs a should_offload method).

Choose where results are stored:

• FileSystemToolResultStore writes results to the local file system.
• Implement the ToolResultStore protocol to target other backends.
• For request-scoped stores in server settings, pass a store per run via the Agent's hook_context.

What is offloaded:

• Only successful, text results are offloaded. Error results (including before_tool human-in-the-loop rejections)
  are left in context.
• Non-text results (image or file content) are left in context; supporting only text is a deliberate choice for now.
  A warning is logged when a non-text result has a matching offload policy.
• Each result is offloaded at most once.

from typing import Annotated

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.hooks.tool_result_offloading import (
    FileSystemToolResultStore,
    OffloadOverChars,
    ToolResultOffloadHook,
)
from haystack.tools import tool


@tool
def search(query: Annotated[str, "The search query"]) -> str:
    """Search the web and return the (potentially large) results."""
    return f"... large result for {query} ..."


offload_hook = ToolResultOffloadHook(
    store=FileSystemToolResultStore(root="tool_results"),
    offload_strategies={"*": OffloadOverChars(4000)},
)
agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
    tools=[search],
    hooks={"after_tool": [offload_hook]},
)
agent.run(messages=[ChatMessage.from_user("Summarize today's tech news")])

How did you test it?

Added new tests.

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
haystack-docs	Ignored	Preview	Jul 3, 2026 12:06pm

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/hooks/tool_result_offloading
__init__.py
hooks.py					223
policies.py
stores.py
haystack/hooks/tool_result_offloading/types
__init__.py
protocol.py					39, 44
Project Total

This report was generated by python-coverage-comment-action