Add AWS Bedrock Converse/ConverseStream provider support #255
Conversation
This commit implements full support for AWS Bedrock's Converse and ConverseStream APIs in ECA.
Key Features:
- Standard chat completions via Converse API
- Streaming responses via ConverseStream API
- Full tool use support with proper formatting
- Custom binary event stream parser (no AWS SDK required)
- Bearer token authentication via external proxy
- Model aliasing for convenience
- Support for additional model parameters (e.g., top_k, topP)
Implementation Details:
- Uses hato.client for HTTP requests (consistent with other providers)
- Custom binary stream parser following AWS Event Stream protocol
- Proper CRC checksum handling for streaming responses
- Comprehensive error handling and logging
Testing:
- 8 tests covering all major functionality (17 assertions)
- Tool use workflows
- Binary stream parsing
- Response parsing
- Payload building
Configuration Example:
```clojure
{:providers
{:bedrock
{:api "bedrock"
:key "${env:BEDROCK_API_KEY}"
:url "https://proxy.example.com/model/{modelId}/converse"
:region "us-east-1"
:models {:claude-3-sonnet
{:modelName "anthropic.claude-3-sonnet-20240229-v1:0"}}}}}
```
Usage:
```clojure
;; Standard request
(provider/request bedrock-config messages {:temperature 0.5 :top_k 200})
;; With tools
(provider/request bedrock-config messages {:tools [tool-spec] :temperature 0.7})
;; Streaming
(provider/request bedrock-stream-config messages {:temperature 0.7})
```
Files Changed:
- src/eca/llm_api.clj: Added require for aws-bedrock provider
- src/eca/llm_providers/aws_bedrock.clj: New provider implementation
- test/eca/llm_providers/aws_bedrock_test.clj: Comprehensive test suite
- AWS_BEDROCK_EXAMPLE.md: Usage documentation
This implementation follows the established patterns in the codebase and is ready for production use.
🤖 Generated with [eca](https://eca.dev)
Co-Authored-By: eca <[email protected]>
CsBigDataHub
force-pushed
the
feature/support-aws-bedrock-provider
branch
from
d8a767c to
08948b1
Compare
|
Future feature request - This Implementation uses Bearer token retrieved externally but we can support AWS Bedrock requires SigV4 signing. Right now I have python code to perform single sign on and retrieve the token which I intend to use here. See here - karthink/gptel#1200 (comment) We can add AWS SigV4 signing directly (using a library like cognitect/aws-api) but we need to add aws library. I have conflicting thoughts on this one. |
|
Maybe we can leverage this after merge - #249. |
|
|
||
| (defn message->bedrock [msg] | ||
| (case (:role msg) | ||
| "tool" |
There was a problem hiding this comment.
is this really working? it should be tool_call right?
Also there are messages missing like reason, ref
|
@CsBigDataHub is this really ready to be reviewed or you are working on it? taking a quick look it seems to be missing features, llm-api integration etc |
|
Hi @ericdallo , no it is not ready, I need more time. |
| This implementation uses Bearer token authentication, which requires | ||
| an external proxy/gateway that handles AWS SigV4 signing. |
There was a problem hiding this comment.
What is this "external proxy" thing?
Won't the implementation work with standard Bedrock APIs, such as https://bedrock-runtime.eu-west-2.amazonaws.com/model/<model_id>/converse ?
There was a problem hiding this comment.
Will work, but you will need to get the token and configure it.
There was a problem hiding this comment.
Yeah, I assumed that the user will create a Bedrock API token first.
But what's the difference when using proxy? Is that for cases when the user has configured their aws credentials on their machine but has no explicit Bedrock API token?
4 participants
Hello @ericdallo,
This commit implements full support for AWS Bedrock's Converse and ConverseStream APIs in ECA.
Key Features:
Implementation Details:
Testing:
Files Changed:
Will add to add to the changelog once I am able to test, my company proxy and cloudflare seems to be blockers at this point.