Skip to content

windower.from_shift_jis and windower.to_shift_jis do not consider latin extended characters #2472

@ariehcore

Description

@ariehcore

Copy pasting this from Windower discord #development channel (here's the message link for conversation context)

I'm having an issue where umlaut characters (and other latin extended characters, i.e. äàöéüè) are not written into text files correctly and I cannot tell what encoding that are going to/from
trying to figure out how to parse chat messages but i am not sure at all how to check the current encoding of a chat message

already tried windower.from_shift_jis and windower.convert_auto_trans after splitting the chat message up, and i cannot tell what encoding it is + I kept receiving errors like the following:
'shift_jis' codec can't encode character '\xe4' in position 22: illegal multibyte sequence
which led me to believe chat handles characters using some form of shift_jis encoding

After checking with Arcon, seems like there is an error in the handling of latin extended characters (umlauts etc) within the shift_jis parser Windower uses.

windower.add_to_chat(207, windower.to_shift_jis('äöüÄÖÜß')) also did not work from Arcon's testing.

These are all parsed correctly ingame, so would be good if they were able to be parsed correctly also.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions