Skip to content

fix: Update HITL tutorial to work with haystack 3.0 #475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sjrl merged 1 commit into from
Jul 20, 2026
Merged

fix: Update HITL tutorial to work with haystack 3.0 #475

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion index.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ notebook = "47_Human_in_the_Loop_Agent.ipynb"
aliases = []
completion_time = "20 min"
created_at = 2025-10-30
dependencies = ["haystack-ai>=2.23.0", "rich"]
dependencies = ["haystack-ai>=3.0.0", "rich"]
featured = true

[[tutorial]]
Expand Down
137 changes: 12 additions & 125 deletions tutorials/47_Human_in_the_Loop_Agent.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,36 +109,17 @@
"metadata": {
"id": "7b2579dc-2c59-4c3b-9691-73f5803d2add"
},
"source": [
"## Setup and Imports\n",
"\n",
"We begin by importing Haystack classes, UI helpers, and the experimental Agent component."
]
"source": "## Setup and Imports\n\nWe begin by importing Haystack classes, UI helpers, the `Agent` component, and the human-in-the-loop confirmation tools."
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"id": "541cf667-a6b6-4e5b-beb1-65745c3f6ac2",
"metadata": {
"id": "541cf667-a6b6-4e5b-beb1-65745c3f6ac2"
},
"outputs": [],
"source": [
"from haystack.components.generators.chat import OpenAIChatGenerator\n",
"from haystack.dataclasses import ChatMessage\n",
"from haystack.tools import create_tool_from_function\n",
"from rich.console import Console\n",
"\n",
"from haystack.components.agents.agent import Agent\n",
"from haystack.human_in_the_loop import (\n",
" AlwaysAskPolicy,\n",
" AskOncePolicy,\n",
" BlockingConfirmationStrategy,\n",
" NeverAskPolicy,\n",
" RichConsoleUI,\n",
" SimpleConsoleUI,\n",
")"
]
"source": "from haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.dataclasses import ChatMessage\nfrom haystack.tools import create_tool_from_function\nfrom rich.console import Console\n\nfrom haystack.components.agents.agent import Agent\nfrom haystack.hooks.human_in_the_loop import (\n AlwaysAskPolicy,\n AskOncePolicy,\n BlockingConfirmationStrategy,\n ConfirmationHook,\n NeverAskPolicy,\n RichConsoleUI,\n SimpleConsoleUI,\n)"
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -195,70 +176,25 @@
"metadata": {
"id": "5c2e21f4-b84f-4411-a5f1-7312afd44188"
},
"source": [
"## Instantiate the Agent\n",
"\n",
"Instantiate the experimental [`Agent`](https://github.com/deepset-ai/haystack-experimental/blob/b2abe5486ee4ad03b6f3c136ca54aee482b9ed01/haystack_experimental/components/agents/agent.py#L69) with multiple tools and assign each tool a **confirmation strategy**."
]
"source": "## Instantiate the Agent\n\nInstantiate the [`Agent`](https://docs.haystack.deepset.ai/docs/agent) with multiple tools. To add human-in-the-loop behavior, create a [`ConfirmationHook`](https://docs.haystack.deepset.ai/docs/confirmationhook) that assigns each tool a **confirmation strategy**, and register it on the Agent's `before_tool` hook point."
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"id": "b6c81b8b-437e-4234-a6e8-8704708f5df9",
"metadata": {
"id": "b6c81b8b-437e-4234-a6e8-8704708f5df9"
},
"outputs": [],
"source": [
"cons = Console()\n",
"\n",
"agent = Agent(\n",
" chat_generator=OpenAIChatGenerator(model=\"gpt-4.1-mini\"),\n",
" tools=[balance_tool, addition_tool, phone_tool],\n",
" system_prompt=\"You are a helpful financial assistant. Use the provided tools to answer user questions.\",\n",
" confirmation_strategies={\n",
" balance_tool.name: BlockingConfirmationStrategy(\n",
" confirmation_policy=AlwaysAskPolicy(), confirmation_ui=RichConsoleUI(console=cons)\n",
" ),\n",
" addition_tool.name: BlockingConfirmationStrategy(\n",
" confirmation_policy=NeverAskPolicy(), confirmation_ui=RichConsoleUI(console=cons)\n",
" ),\n",
" phone_tool.name: BlockingConfirmationStrategy(\n",
" confirmation_policy=AskOncePolicy(), confirmation_ui=RichConsoleUI(console=cons)\n",
" ),\n",
" },\n",
")"
]
"source": "cons = Console()\n\nconfirmation_hook = ConfirmationHook(\n confirmation_strategies={\n balance_tool.name: BlockingConfirmationStrategy(\n confirmation_policy=AlwaysAskPolicy(), confirmation_ui=RichConsoleUI(console=cons)\n ),\n addition_tool.name: BlockingConfirmationStrategy(\n confirmation_policy=NeverAskPolicy(), confirmation_ui=RichConsoleUI(console=cons)\n ),\n phone_tool.name: BlockingConfirmationStrategy(\n confirmation_policy=AskOncePolicy(), confirmation_ui=RichConsoleUI(console=cons)\n ),\n }\n)\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4.1-mini\"),\n tools=[balance_tool, addition_tool, phone_tool],\n system_prompt=\"You are a helpful financial assistant. Use the provided tools to answer user questions.\",\n hooks={\"before_tool\": [confirmation_hook]},\n)"
},
{
"cell_type": "markdown",
"id": "4a1e1fc9-bf38-4207-a50f-dded827c4ae6",
"metadata": {
"id": "4a1e1fc9-bf38-4207-a50f-dded827c4ae6"
},
"source": [
"### Explanation\n",
"\n",
"Each [`BlockingConfirmationStrategy`](https://github.com/deepset-ai/haystack-experimental/blob/b2abe5486ee4ad03b6f3c136ca54aee482b9ed01/haystack_experimental/components/agents/human_in_the_loop/strategies.py#L32) defines when and how the agent asks for confirmation before executing a tool:\n",
"\n",
"* **`AlwaysAskPolicy`** – Always asks for approval before running.\n",
"* **`NeverAskPolicy`** – Executes automatically without user confirmation.\n",
"* **`AskOncePolicy`** – Asks once per tool, remembers approval for future runs.\n",
"\n",
"The UI can be either:\n",
"\n",
"* `RichConsoleUI` for colorized and more aesthetic prompts.\n",
"* `SimpleConsoleUI` for basic prompts.\n",
"\n",
"\n",
"Both UIs will present three options:\n",
"\n",
"- **y**: Proceed with the tool execution as planned\n",
"- **n**: Cancel the tool execution; agent will try alternative approaches\n",
"- **m**: Edit the tool parameters before execution\n",
"\n",
"**NOTE**: Custom UIs and Policies can also be implemented by following the respective `ConfirmationUI` and `ConfirmationPolicy` protocols. This allows full flexibility for domain-specific workflows."
]
"source": "### Explanation\n\nThe `ConfirmationHook` runs at the Agent's `before_tool` hook point, after the model requests a tool call but before the tool actually runs. For each requested tool, it looks up the matching entry in `confirmation_strategies`. A key can be a single tool name, a tuple of tool names sharing one strategy, or the wildcard `\"*\"` which applies to any tool without a more specific entry.\n\nEach [`BlockingConfirmationStrategy`](https://docs.haystack.deepset.ai/docs/blockingconfirmationstrategy) defines when and how the agent asks for confirmation before executing a tool. The `confirmation_policy` decides *when* to ask:\n\n* **`AlwaysAskPolicy`** – Always asks for approval before running.\n* **`NeverAskPolicy`** – Executes automatically without user confirmation.\n* **`AskOncePolicy`** – Asks once per tool, remembers approval for future runs.\n\nThe `confirmation_ui` decides *how* to ask:\n\n* `RichConsoleUI` for colorized and more aesthetic prompts.\n* `SimpleConsoleUI` for basic prompts.\n\n\nBoth UIs will present three options:\n\n- **y**: Proceed with the tool execution as planned\n- **n**: Cancel the tool execution; agent will try alternative approaches\n- **m**: Edit the tool parameters before execution\n\n**NOTE**: Custom UIs and Policies can also be implemented by following the respective `ConfirmationUI` and `ConfirmationPolicy` protocols. This allows full flexibility for domain-specific workflows."
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -572,32 +508,13 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"id": "2c650eb7-244d-4a58-b024-e79a1b79e4cd",
"metadata": {
"id": "2c650eb7-244d-4a58-b024-e79a1b79e4cd"
},
"outputs": [],
"source": [
"from haystack.human_in_the_loop.types import ConfirmationPolicy\n",
"from typing import Any\n",
"\n",
"\n",
"class BudgetBasedPolicy(ConfirmationPolicy):\n",
" \"\"\"Ask for confirmation when operations exceed a cost threshold.\"\"\"\n",
"\n",
" def __init__(self, cost_threshold: float = 10.0):\n",
" self.cost_threshold = cost_threshold\n",
"\n",
" def should_ask(self, tool_name: str, tool_description: str, tool_params: dict[str, Any]) -> bool:\n",
" \"\"\"Ask if the operation cost exceeds the threshold.\"\"\"\n",
" # Check for cost-related parameters\n",
" cost = tool_params.get(\"cost\", 0.0)\n",
" amount = tool_params.get(\"amount\", 0.0)\n",
" price = tool_params.get(\"price\", 0.0)\n",
"\n",
" return max(cost, amount, price) > self.cost_threshold"
]
"source": "from haystack.hooks.human_in_the_loop.types import ConfirmationPolicy\nfrom typing import Any\n\n\nclass BudgetBasedPolicy(ConfirmationPolicy):\n \"\"\"Ask for confirmation when operations exceed a cost threshold.\"\"\"\n\n def __init__(self, cost_threshold: float = 10.0):\n self.cost_threshold = cost_threshold\n\n def should_ask(self, tool_name: str, tool_description: str, tool_params: dict[str, Any]) -> bool:\n \"\"\"Ask if the operation cost exceeds the threshold.\"\"\"\n # Check for cost-related parameters\n cost = tool_params.get(\"cost\", 0.0)\n amount = tool_params.get(\"amount\", 0.0)\n price = tool_params.get(\"price\", 0.0)\n\n return max(cost, amount, price) > self.cost_threshold"
},
{
"cell_type": "markdown",
Expand All @@ -613,43 +530,13 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"id": "7810bdd3-7592-412e-a15b-106b5d4af44e",
"metadata": {
"id": "7810bdd3-7592-412e-a15b-106b5d4af44e"
},
"outputs": [],
"source": [
"from haystack.components.generators.utils import print_streaming_chunk\n",
"\n",
"\n",
"def expense(cost: float, description: str) -> float:\n",
" \"\"\"Submit an expense report that has a `cost` and `description`\"\"\"\n",
" # This is where we would add a real submission request\n",
" return \"Expense report submitted successfully!\"\n",
"\n",
"\n",
"expense_tool = create_tool_from_function(\n",
" function=expense, name=\"expense\", description=\"Submit an expense report that has a `cost` and `description`\"\n",
")\n",
"\n",
"cons = Console()\n",
"\n",
"agent = Agent(\n",
" chat_generator=OpenAIChatGenerator(model=\"gpt-4.1\"),\n",
" tools=[expense_tool],\n",
" system_prompt=(\n",
" \"You are a helpful financial assistant that can submit expense reports for users. \"\n",
" \"Use the `expense` tool which only requires a cost amount and short description of the expense (e.g. 'busines lunch'). \"\n",
" \"Only respond with whether the report was submitted successfully. \"\n",
" ),\n",
" confirmation_strategies={\n",
" expense_tool.name: BlockingConfirmationStrategy(\n",
" confirmation_policy=BudgetBasedPolicy(cost_threshold=100.0), confirmation_ui=RichConsoleUI(console=cons)\n",
" )\n",
" },\n",
")"
]
"source": "from haystack.components.generators.utils import print_streaming_chunk\n\n\ndef expense(cost: float, description: str) -> float:\n \"\"\"Submit an expense report that has a `cost` and `description`\"\"\"\n # This is where we would add a real submission request\n return \"Expense report submitted successfully!\"\n\n\nexpense_tool = create_tool_from_function(\n function=expense, name=\"expense\", description=\"Submit an expense report that has a `cost` and `description`\"\n)\n\ncons = Console()\n\nconfirmation_hook = ConfirmationHook(\n confirmation_strategies={\n expense_tool.name: BlockingConfirmationStrategy(\n confirmation_policy=BudgetBasedPolicy(cost_threshold=100.0), confirmation_ui=RichConsoleUI(console=cons)\n )\n }\n)\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4.1\"),\n tools=[expense_tool],\n system_prompt=(\n \"You are a helpful financial assistant that can submit expense reports for users. \"\n \"Use the `expense` tool which only requires a cost amount and short description of the expense (e.g. 'busines lunch'). \"\n \"Only respond with whether the report was submitted successfully. \"\n ),\n hooks={\"before_tool\": [confirmation_hook]},\n)"
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -857,4 +744,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}