Batch inference — OpenAI /v1/batches & Anthropic /v1/messages/batches

[JGoutin]

Implement batch inference APIs backed by Amazon Bedrock batch inference.

OpenAI-compatible

• POST /v1/batches
• GET /v1/batches
• GET /v1/batches/{batch_id}
• POST /v1/batches/{batch_id}/cancel

Anthropic-compatible

• POST /v1/messages/batches
• GET /v1/messages/batches
• GET /v1/messages/batches/{message_batch_id}
• DELETE /v1/messages/batches/{message_batch_id}
• POST /v1/messages/batches/{message_batch_id}/cancel
• GET /v1/messages/batches/{message_batch_id}/results

REF:

• https://platform.openai.com/docs/api-reference/batch
• https://docs.anthropic.com/en/api/creating-message-batches
• https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html