Skip to content
This repository was archived by the owner on Nov 19, 2025. It is now read-only.
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/user-guide/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
.. toctree::
:maxdepth: 2

reinforce.rst
sft.rst
knowledge-distillation.rst
dpo.rst
Expand All @@ -19,6 +20,9 @@
:ref:`Prerequisite Obtaining a Pre-Trained Model <prerequisite>`
This section provides instructions on how to download pre-trained LLMs in .nemo format. The following section will use these base LLMs for further fine-tuning and alignment.

:ref:`Model Alignment by REINFORCE <nemo-aligner-reinforce>`
In this tutorial, we will guide you through the process of aligning a NeMo Framework model using REINFORCE. This method can be applied to various models, including LLaMa2 and Mistral, with our scripts functioning consistently across different models.

:ref:`Model Alignment by Supervised Fine-Tuning (SFT) <nemo-aligner-sft>`
In this section, we walk you through the most straightforward alignment method. We use a supervised dataset in the prompt-response pairs format to fine-tune the base model according to the desired behavior.

Expand Down Expand Up @@ -59,6 +63,14 @@
- Mistral
- Nemotron-4
- Mixtral
* - :ref:`REINFORCE <nemo-aligner-reinforce>`
- Yes
- Yes
- Yes
- Yes (✓)
- Yes
- Yes
-
* - :ref:`SFT <nemo-aligner-sft>`
-
- Yes (✓)
Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/reinforce.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.. include:: /content/nemo.rsts

.. _model-aligner-reinforce:
.. _nemo-aligner-reinforce:

Model Alignment by REINFORCE
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Expand Down
Loading