diff --git a/docs-website/docs/pipeline-components/generators/azureopenaichatgenerator.mdx b/docs-website/docs/pipeline-components/generators/azureopenaichatgenerator.mdx index d886817beb..8a92db84c7 100644 --- a/docs-website/docs/pipeline-components/generators/azureopenaichatgenerator.mdx +++ b/docs-website/docs/pipeline-components/generators/azureopenaichatgenerator.mdx @@ -59,6 +59,12 @@ Then, the component needs a list of `ChatMessage` objects to operate. `ChatMessa You can pass any chat completion parameters that are valid for the `openai.ChatCompletion.create` method directly to `AzureOpenAIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the supported parameters, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference). +:::tip[Production cost control] + +For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside an agent or retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size. + +::: + You can also specify a model for this component through the `azure_deployment` init parameter. ### Structured Output diff --git a/docs-website/docs/pipeline-components/generators/azureopenaigenerator.mdx b/docs-website/docs/pipeline-components/generators/azureopenaigenerator.mdx index 7060918141..295595b770 100644 --- a/docs-website/docs/pipeline-components/generators/azureopenaigenerator.mdx +++ b/docs-website/docs/pipeline-components/generators/azureopenaigenerator.mdx @@ -45,6 +45,12 @@ We recommend using environment variables instead of initialization parameters. Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the supported parameters, refer to the [Azure documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference). +:::tip[Production cost control] + +For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside a retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size. + +::: + You can also specify a model for this component through the `azure_deployment` init parameter. ### Streaming diff --git a/docs-website/docs/pipeline-components/generators/openaichatgenerator.mdx b/docs-website/docs/pipeline-components/generators/openaichatgenerator.mdx index e4b7b47bd9..807509cadd 100644 --- a/docs-website/docs/pipeline-components/generators/openaichatgenerator.mdx +++ b/docs-website/docs/pipeline-components/generators/openaichatgenerator.mdx @@ -37,6 +37,12 @@ Then, the component needs a list of `ChatMessage` objects to operate. `ChatMessa You can pass any chat completion parameters valid for the `openai.ChatCompletion.create` method directly to `OpenAIChatGenerator` using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat). +:::tip[Production cost control] + +For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside an agent or retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size. + +::: + `OpenAIChatGenerator` can support custom deployments of your OpenAI models through the `api_base_url` init parameter. ### Structured Output diff --git a/docs-website/docs/pipeline-components/generators/openaigenerator.mdx b/docs-website/docs/pipeline-components/generators/openaigenerator.mdx index 0c6cf0e7d0..86c793c0c8 100644 --- a/docs-website/docs/pipeline-components/generators/openaigenerator.mdx +++ b/docs-website/docs/pipeline-components/generators/openaigenerator.mdx @@ -35,6 +35,12 @@ generator = OpenAIGenerator(api_key=Secret.from_token(""), model=" Then, the component needs a prompt to operate, but you can pass any text generation parameters valid for the `openai.ChatCompletion.create` method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the OpenAI API, refer to the [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat). +:::tip[Production cost control] + +For production workloads, set a completion token limit such as `generation_kwargs={"max_completion_tokens": 512}`. This bounds response length, latency, and API spend. If you use this generator inside a retrying pipeline, the limit applies to each model call, so choose a value that fits your expected answer size. + +::: + `OpenAIGenerator` supports custom deployments of your OpenAI models through the `api_base_url` init parameter. ### Streaming