[BUG] - Message length mismatch for non ASCII-characters

[FloEdelmann]

UE Version

UE 5.3.2.0

Frontend Version

• @epicgames-ps/lib-pixelstreamingcommon-ue5.5: 0.3.2
• @epicgames-ps/lib-pixelstreamingfrontend-ue5.5: 1.2.5

Problem component

Frontend

Description

When sending a UIInteraction via stream.emitUIInteraction({…}) containing a non-ASCII character like a German umlaut (äöüÄÖÜß), the message cannot be parsed correctly on the Unreal side. After further investigation, I noticed that this is because a mismatch of different string length calculation methods:

To determine the initial buffer size, TextEncoder.encode's length is used:

PixelStreamingInfrastructure/Frontend/library/src/UeInstanceMessage/SendMessageController.ts

Lines 91 to 94 in 067138e

	// 2 bytes for string length
	byteLength += 2;
	// 2 bytes per characters
	byteLength += 2 * textEncoder.encode(element as string).length;

But later, when the string is written into the buffer, raw String.length is used:

PixelStreamingInfrastructure/Frontend/library/src/UeInstanceMessage/SendMessageController.ts

Lines 132 to 137 in 067138e

	data.setUint16(byteOffset, (element as string).length, true);
	byteOffset += 2;
	for (let i = 0; i < (element as string).length; i++) {
	data.setUint16(byteOffset, (element as string).charCodeAt(i), true);
	byteOffset += 2;
	}

For ASCII characters, this length is the same, but for e.g. umlauts, the length differs:

new TextEncoder().encode('a')         // → Uint8Array [ 97 ]
new TextEncoder().encode('a').length  // → 1
'a'.charCodeAt(0)                     // → 97
'a'.length                            // → 1

new TextEncoder().encode('ü')         // → Uint8Array [ 195, 188 ]
new TextEncoder().encode('ü').length  // → 2
'ü'.charCodeAt(0)                     // → 252
'ü'.length                            // → 1

This means that more space is reserved in the buffer than is actually written, so trailing null bytes are sent in the message, which cause the Unreal-side C++ JSON parser to trip.

Steps to Reproduce

Open a stream and call stream.emitUIInteraction({ "foo": "bär" }).

Expected behavior

The JSON object { "foo": "bär" } is deserialized correctly in Unreal.

Suggested fix

Use raw String.length anywhere, i.e. replace

PixelStreamingInfrastructure/Frontend/library/src/UeInstanceMessage/SendMessageController.ts

Line 94 in 067138e

byteLength += 2 * textEncoder.encode(element as string).length;

with

                    byteLength += 2 * (element as string).length;

Workaround

Instead of calling stream.emitUIInteraction({ "foo": "bär" }), we can implement the logic in client-side code:

const jsonString = JSON.stringify({ "foo": "bär" })
const toStreamerMessages = stream.webRtcController.streamMessageController.toStreamerMessages

const messageType = 'UIInteraction'
const messageFormat = toStreamerMessages.get(messageType)
if (messageFormat === undefined) {
    throw new Error(`streaming message type ${messageType} is not available`)
}

const data = new DataView(new ArrayBuffer(3 + jsonString.length * 2))
data.setUint8(0, messageFormat.id)
data.setUint16(1, jsonString.length, true)
for (let i = 0; i < jsonString.length; i++) {
    data.setUint16(3 + i * 2, jsonString.charCodeAt(i), true)
}

stream.webRtcController.dataChannelSender.sendData(data.buffer)