Read timeout with Autogen Bedrock implementation in v0.2.40 #5505
Description
We are facing read timeout error while using the autogen bedrock implementation with large input tokens. The default read timeout in boto3 configuration is 60 seconds and autogen implementation doesn't allow any modification to this value while initializing the BedrockClient class.
Here is the error message:
RuntimeError: Failed to get response from Bedrock: Read timeout on endpoint URL: "https://bedrock-runtime.us-west-2.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse"
We can see that the __init__ method for class BedrockClient does accept variable keyword arguments however the same is not being used while defining the variable bedrock_config within this method.
def __init__(self, **kwargs: Any):
"""
Initialises BedrockClient for Amazon's Bedrock Converse API
"""
self._aws_access_key = kwargs.get("aws_access_key", None)
self._aws_secret_key = kwargs.get("aws_secret_key", None)
self._aws_session_token = kwargs.get("aws_session_token", None)
self._aws_region = kwargs.get("aws_region", None)
self._aws_profile_name = kwargs.get("aws_profile_name", None)
if not self._aws_access_key:
self._aws_access_key = os.getenv("AWS_ACCESS_KEY_ID")
if not self._aws_secret_key:
self._aws_secret_key = os.getenv("AWS_SECRET_ACCESS_KEY")
if not self._aws_session_token:
self._aws_session_token = os.getenv("AWS_SESSION_TOKEN")
if not self._aws_region:
self._aws_region = os.getenv("AWS_DEFAULT_REGION")
if self._aws_region is None:
raise ValueError("Region is required to use the Amazon Bedrock API.")
# Initialize Bedrock client, session, and runtime
bedrock_config = Config(
region_name=self._aws_region,
signature_version="v4",
retries={"max_attempts": self._retries, "mode": "standard"},
)
session = boto3.Session(
aws_access_key_id=self._aws_access_key,
aws_secret_access_key=self._aws_secret_key,
aws_session_token=self._aws_session_token,
profile_name=self._aws_profile_name,
)
self.bedrock_runtime = session.client(service_name="bedrock-runtime", config=bedrock_config)