Prerequisites
- uv package manager (
brew install uv)
Run baseline QWEN model: uv run src/twenty_questions/qwen_player.py --url http://twenty-questions-alb-1736724246.eu-north-1.elb.amazonaws.com:8000/
Run Claude 3.5 Sonnet: uv run src/twenty_questions/qwen_player.py --url http://twenty-questions-alb-1736724246.eu-north-1.elb.amazonaws.com:8000/ --model claude
Generate dataset: uv run src/twenty_questions/dataset_generator.py
Note that each time you run the dataset generator, three "games" will be played against the same noun. The first game will be played by QWEN (bad generation of questions) and the following two games will be played by Claude 3.5 Sonnet (good generation of questions).
Each time you run it, rows will be appended to the "game_results.csv" file.
deepspeed --num_gpus 4 src/finetune/qwen_finetune.py