Ignore UTF-8 surrogate characters when encoding request content

[nguyenhoan1988]

Summary

• Add tests asserting surrogate code points are dropped from request bodies and JSON.
• Change all internal UTF-8 encodings to use errors="ignore" to avoid "surrogates not allowed" Unicode errors.

Checklist

• I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.

[cclauss]

Please rebase to pick up the fixes merged in:

• Upgrade Python formatter ruff #3651
• Upgrade Python type checker mypy #3654

[lovelydinosaur]

I understand that this PR may be closed in case there was no previous discussion.

Thank you, no. I wouldn't consider errors='ignore' an improvement here.