Failed to reproduce

[xjtu-xsbsy]

Hello, thank you for your great work. However, when I reproduced the Andreasson_2020_ribozyme.csv experiment from fitness_processed_assays.zip using the script under Nucleotide_Transformer, the calculated scores differed significantly from the corresponding results in model_predictions.zip.

[murfalo]

Hello! Thanks for bringing this to our attention. We expect to be able to address this in the next few weeks. It is possible the nucleotide transformer script has not been updated with the scoring method we used in the paper.

In the mean time, could you provide details as to:

1. Which nucleotide transformer model/checkpoint you used, and
2. Provide the conflicting values so we can understand how/the degree to which they differ?

Thanks!

[xjtu-xsbsy]

Hello！The transformer model/checkpoint is nucleotide-transformer-2.5b-multi-species. For Andreasson_2020_ribozyme, the value of “C10A,U56C” calculated using the compute_fitness.py is -4.314168930053711, but the value in “model_predictions/NT” is -14.57473514717363.
Thanks!