Skip to content

Conversation

@alexandremuzio
Copy link
Contributor

Initial working version of MRT.
Besides the difference in the learning procedure, most of the MRT types are pretty similar to the PPO ones, such as MRTRLElement. The main difference is that MRT generates a batch of candidate sentences for the same prompt instead of a single sentence per prompt.

Main changes:

  • Add AccelerateMRTTrainer
  • Introducing MRT configs/data types (trlx/data/mrt_types.py,
  • Example for translation: examples/mrt_translation_t5.py

I've currently tested it with t5 on translation and summarization tasks.

TODOs:

  • Test on decoder-only models
  • Make sure it is working for
  • Add support for MarianMT models
  • debug_mrt_summarize_daily_cnn_t5.py (only for debugging)

Please let me know if there are any other suggestions as well. cc @LouisCastricato

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant