Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ Then, the component needs a list of `ChatMessage` objects to operate. `ChatMessa

You can pass any chat completion parameters that are valid for the `openai.ChatCompletion.create` method directly to `AzureOpenAIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the supported parameters, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).

:::tip[Production cost control]

For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside an agent or retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size.

:::

You can also specify a model for this component through the `azure_deployment` init parameter.

### Structured Output
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,12 @@ We recommend using environment variables instead of initialization parameters.

Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the supported parameters, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference).

:::tip[Production cost control]

For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside a retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size.

:::

You can also specify a model for this component through the `azure_deployment` init parameter.

### Streaming
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ Then, the component needs a list of `ChatMessage` objects to operate. `ChatMessa

You can pass any chat completion parameters valid for the `openai.ChatCompletion.create` method directly to `OpenAIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).

:::tip[Production cost control]

For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside an agent or retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size.

:::

`OpenAIChatGenerator` can support custom deployments of your OpenAI models through the `api_base_url` init parameter.

### Structured Output
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ generator = OpenAIGenerator(api_key=Secret.from_token("<your-api-key>"), model="

Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).

:::tip[Production cost control]

For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside a retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size.

:::

`OpenAIGenerator` supports custom deployments of your OpenAI models through the `api_base_url` init parameter.

### Streaming
Expand Down