Skip to content

Remove number validation on deserialization for DynamoDB resources#4699

Open
jonathan343 wants to merge 1 commit intodevelopfrom
dynamodb-resource-deserialization-bugfix
Open

Remove number validation on deserialization for DynamoDB resources#4699
jonathan343 wants to merge 1 commit intodevelopfrom
dynamodb-resource-deserialization-bugfix

Conversation

@jonathan343
Copy link
Contributor

@jonathan343 jonathan343 commented Jan 14, 2026

Important

High-level resources in boto3 are feature frozen, however, this is a bugfix that addresses a valid use case that is currently broken.

Summay

This PR removes client-side number validation during deserialization for DynamoDB resources. If DynamoDB stored and returned a value, client-side validation is unnecessary overhead and can result in customers being unable to parse valid numbers.

Addresses: #2500, #4693

Background

Both serialization and deserialization currently use DYNAMODB_CONTEXT.create_decimal() to validate numbers:

Serialization:

def _serialize_n(self, value):
number = str(DYNAMODB_CONTEXT.create_decimal(value))
if number in ['Infinity', 'NaN']:
raise TypeError('Infinity and NaN not supported')
return number

Deserialization:

def _deserialize_n(self, value):
return DYNAMODB_CONTEXT.create_decimal(value)

This works for most customers because numbers that pass serialization will also pass deserialization. However, data that doesn't flow through the dynamodb deserialization logic for resources can fail to deserialize. Examples include data written by other SDKs (Go, Java, etc.), via the low-level client, or using UpdateExpression and ExpressionAttributeValues as shown below:

table.update_item(
    Key={'p': p}, 
    UpdateExpression='SET a = :val',
    ExpressionAttributeValues={':val': Decimal("1e100"}
)

Root Cause

DynamoDB limits numbers to 38 significant digits, while Python's Decimal context counts all digits including trailing zeros. For example:

Number Significant Digits Total Digits DynamoDB boto3 (before)
1234567895171680000000000000000000000000 16 40 Valid Rounded exception
1E+100 1 1 Valid Valid

This mismatch causes valid DynamoDB numbers to raise decimal.Rounded during deserialization.

@nyh
Copy link

nyh commented Jan 14, 2026

Thanks. Please note that the explanation that this situation can only arise if the item is written by a different library is not accurate. In the issue I created I had the following example, where the write was also done in boto3, and worked - only the read failed:

table.update_item(Key={'p': p},  UpdateExpression='SET a = :val',
            ExpressionAttributeValues={':val': Decimal("1e100"})

@jonathan343
Copy link
Contributor Author

Thanks. Please note that the explanation that this situation can only arise if the item is written by a different library is not accurate. In the issue I created I had the following example, where the write was also done in boto3, and worked - only the read failed:

table.update_item(Key={'p': p},  UpdateExpression='SET a = :val',
            ExpressionAttributeValues={':val': Decimal("1e100"})

Right, thanks! I updated the description to mention this case.

@rowanseymour
Copy link

I think we still want to remove the Rounded trap from DYNAMODB_CONTEXT because now you have an odd situation where you can retrieve Decimal('1234567895171680000000000000000000000000') but you can't re-serialize it.

I think the remaining Inexact trap will prevent sending invalid values off to DynamoDB but some good unit tests would confirm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments