Remove number validation on deserialization for DynamoDB resources#4699
Remove number validation on deserialization for DynamoDB resources#4699jonathan343 wants to merge 1 commit intodevelopfrom
Conversation
|
Thanks. Please note that the explanation that this situation can only arise if the item is written by a different library is not accurate. In the issue I created I had the following example, where the write was also done in boto3, and worked - only the read failed: table.update_item(Key={'p': p}, UpdateExpression='SET a = :val',
ExpressionAttributeValues={':val': Decimal("1e100"}) |
Right, thanks! I updated the description to mention this case. |
|
I think we still want to remove the I think the remaining |
Important
High-level resources in boto3 are feature frozen, however, this is a bugfix that addresses a valid use case that is currently broken.
Summay
This PR removes client-side number validation during deserialization for DynamoDB resources. If DynamoDB stored and returned a value, client-side validation is unnecessary overhead and can result in customers being unable to parse valid numbers.
Addresses: #2500, #4693
Background
Both serialization and deserialization currently use
DYNAMODB_CONTEXT.create_decimal()to validate numbers:Serialization:
boto3/boto3/dynamodb/types.py
Lines 213 to 217 in a6ff277
Deserialization:
boto3/boto3/dynamodb/types.py
Lines 288 to 289 in a6ff277
This works for most customers because numbers that pass serialization will also pass deserialization. However, data that doesn't flow through the dynamodb deserialization logic for resources can fail to deserialize. Examples include data written by other SDKs (Go, Java, etc.), via the low-level client, or using
UpdateExpressionandExpressionAttributeValuesas shown below:Root Cause
DynamoDB limits numbers to 38 significant digits, while Python's
Decimalcontext counts all digits including trailing zeros. For example:1234567895171680000000000000000000000000Roundedexception1E+100This mismatch causes valid DynamoDB numbers to raise
decimal.Roundedduring deserialization.