Skip to content

Minor mistakes in file "fra_eng_dataset.py" #1

@changliu01

Description

@changliu01

Hi! First, thank you for your simple but impressive tutorial about Transformers. It helps me a lot not only on my study but also my TA work. However, the output of the function "translate_sentences" is really strange. For example,

Source sentence is: gras jumeaux Appellez-moi tombée Recule opposé vos musulmane Montrez-moi plutôt !
Target sentence is: She followed him home to find out where he lived .
Model translation is: She followed him home to find out where he lived .

Although the translation of the model is the same as the target sentence, the meaning of the source sentence does not seem to correspond to the target sentence. Actually, the source sentence is more like a meaningless sentence, which is really confusing.

After double-checking your code. I think there may be something wrong in the file "fra_eng_dataset.py", specifically in line 74,
fra_token_sentence = [self.eng_token_dict['<START>']] + eng_token_sentence
I modify it to
fra_token_sentence = [self.fra_token_dict['<START>']] + fra_token_sentence
And it works for me.
Here is an example of the output from the aforementioned function "translate_sentences", after my modification:

Source sentence is: Je suis chanceux de vous avoir pour amie .
Target sentence is: I 'm lucky to have you as a friend .
Model translation is: I 'm lucky to have you a friend .

Thank you for your excellent tutorial again! Hope my small advice can help you.

Tips: you should remove the "sentences.pkl" generated before.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions