The issues in scene text editing in a different language

[uijinee]

Hello! We are attempting to perform AR translation from a different language font to English. However, we've encountered the following issue.

• adding alphabets instead of representing the input characters as they are
• the input characters appear in two lines

Do you have any insights into possible reasons for this?

[Question406]

hello, our model may be less effective in this abnormal aspect ratio mask since we are training on ground-truth English character images with more narrow masks.

I think you can try several things to improve the result: 1. Try more random seed and guidance scales. 2. First in-paint the masked part with an inpainting model and then generate the target characters with a less wide mask.

[Neks11]

Hello,

Thanks for sharing your work.
Can results be improved for wider mask, curved mask by fine tuning the model.
Did you try the same?

Also model is not giving good results for exiting Text font style transfer with curved, wider mask.

[Question406]

@Neks11 Hi, the current synthetic data generation pipeline in our work does not generate many authentic curved texts, which is limited by the rendering engine. I think you could try more advanced data augmentation. Another direction worth trying is the Anytext(https://github.com/tyxsspa/anytext) and TextDiffuser(https://jingyechen.github.io/textdiffuser/), which incorporates both text masks and direct glyph-level guidance with character scene graphs.