Skip to content

Comments

fix: memory corruption and serialization issues (v0.5.21)#15

Merged
justrach merged 10 commits intomainfrom
justrach/fix-gitignore-cleanup
Feb 3, 2026
Merged

fix: memory corruption and serialization issues (v0.5.21)#15
justrach merged 10 commits intomainfrom
justrach/fix-gitignore-cleanup

Conversation

@justrach
Copy link
Owner

@justrach justrach commented Feb 3, 2026

Summary

This PR fixes several critical issues discovered when running TurboAPI with free-threaded Python 3.13t and Metal/MLX GPU frameworks:

  • Memory corruption in async handlers - Body bytes were being corrupted between Rust and Python due to improper memory ownership transfer
  • Response object serialization - Response objects were serialized as their string representation instead of actual content
  • Async handler parameter passing - Async handlers with BaseModel params received kwargs instead of the model instance
  • JSON parsing with special characters - simd-json rejected valid JSON that Python's json module accepts

Changes

Core Fixes

  • Use PyBytes::new() instead of as_slice() for proper memory ownership in async handlers
  • Add defensive bytes(bytearray(body)) copy in Python to prevent corruption with Metal/MLX
  • Route async handlers with BaseModel params through enhanced path for proper model instantiation
  • Add Python json.loads fallback when simd-json fails

Response Handling

  • Improve Response object detection via model_dump() and attribute checks
  • Handle binary data (audio/wav) without UnicodeDecodeError
  • Add base64 encoding fallback for binary content

Housekeeping

  • Clean up .gitignore (remove duplicates, improve patterns)
  • Add unit tests for all fixes
  • Add uv.lock for reproducible builds

Test plan

  • Unit tests pass for Response serialization
  • Unit tests pass for async handler classification
  • Unit tests pass for JSON parsing with special characters
  • Manual testing with MLX voice generation server (async handlers with BaseModel params)
  • Both sync and async handlers work correctly with Metal GPU loaded

justrach and others added 10 commits February 3, 2026 14:28
- Remove duplicate benchmark_graphs/ entry
- Change *.json to specific patterns to avoid ignoring config files
- Remove Cargo.lock from ignore (should be tracked for apps)
- Add common Python/IDE patterns

Generated with AI

Co-Authored-By: AI <ai@example.com>
- Check for model_dump() method first (handles dhi/Pydantic models)
- Properly detect Response objects via body/status_code attributes
- Handle binary response bodies without crashing

This fixes Issue 1 where Response objects were being serialized
as their string representation instead of actual content.

Generated with AI

Co-Authored-By: AI <ai@example.com>
- simd-json has stricter parsing than Python's json module
- Fall back to Python's json.loads for edge cases with special chars
- Fixes JSON parsing errors with exclamation marks and unicode

This fixes Issue 3 where certain valid JSON was rejected by simd-json.

Generated with AI

Co-Authored-By: AI <ai@example.com>
- Set needs_body=True for async handlers with BaseModel parameters
- Ensures proper model instantiation in Python context
- Prevents kwargs being passed instead of model instance

This fixes Issue 2 where async handlers received individual kwargs
instead of the validated BaseModel instance.

Generated with AI

Co-Authored-By: AI <ai@example.com>
- Add defensive copy using bytes(bytearray(body)) in parse_json_body
- Fix normalize_response to handle binary data without UnicodeDecodeError
- Add base64 encoding fallback for binary content in make_serializable

When running free-threaded Python (3.13t) with GPU frameworks like MLX,
memory can be corrupted between operations due to concurrent access.
The defensive copy ensures body bytes are in isolated memory before parsing.

Generated with AI

Co-Authored-By: AI <ai@example.com>
- Change from body_vec.as_slice() to PyBytes::new() for proper ownership
- PyBytes copies data into Python-managed memory
- Prevents corruption when Metal/MLX accesses shared memory regions

The previous approach passed a Rust slice reference to Python, which
could be corrupted by GPU memory operations in free-threaded mode.
PyBytes::new() ensures the data is copied to Python's heap.

Generated with AI

Co-Authored-By: AI <ai@example.com>
- Test Response serialization and body extraction
- Test async handler classification with BaseModel params
- Test JSON parsing with special characters and unicode
- Test empty body handling

Generated with AI

Co-Authored-By: AI <ai@example.com>
Generated with AI

Co-Authored-By: AI <ai@example.com>
Generated with AI

Co-Authored-By: AI <ai@example.com>
Release includes:
- Fix Response object serialization
- Fix async handler BaseModel parameter passing
- Add simd-json fallback for special characters
- Fix memory corruption with free-threaded Python + Metal/MLX
- Clean up .gitignore

Generated with AI

Co-Authored-By: AI <ai@example.com>
@justrach justrach merged commit 49b3040 into main Feb 3, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant