MindSLM

Privacy-Centric Small Language Models (SLMs) for Mental Health Therapy

MindSLM is designed to deploy small language models that prioritize user privacy, specifically tailored for mental health therapy applications.

Project Files

• Qwen2_5_Unsloth_2x_faster_finetuning.ipynb
  This notebook provides the training workflow using UnSloth for MindSLM. Adjust the parameters and configurations as needed to suit your specific use case.

• eval.py
  This script handles inference tasks for large language models (LLMs) on the testing dataset.

• trad-eval.py
  This script handles evaluating the models' performance on the testing dataset using traditional evaluation metrics including RougeL and BERTScore, and output the results in CSV files in the eval_outputs/ directory and the plots in the plots/ directory.

• llm_aj_seq.ipynb, llm_aj.py These notebooks contains the code for evaluating the models' performance on the testing dataset using llm-as-a-judge by querying the OpenAI API. The _seq version is for sequential querying, while the other uses the OpenAI Batch API.

Getting Started

1. Clone the repository and ensure all dependencies are installed.

2. Customize the training notebook to match your requirements.

3. Use Few-Shot Prompting

   • Request access to the Psych8k dataset, which is currently gated on Hugging Face: https://huggingface.co/datasets/EmoCareAI/Psych8k
   • Once you have access, run example.py to generate the few-shot examples.
   • Replace the placeholder text in eval.py with the examples to enable few-shot prompting.

4. Use eval.py to generate the trained models' output on the test dataset.

5. Use trad-eval.py to evaluate the models' performance using traditional evaluation metrics.

6. Use llm_aj_seq.ipynb or llm_aj.py to evaluate the models' performance using llm-as-a-judge.

Features

• Privacy-centric architecture designed for mental health applications.
• Optimized for small language models to enable local deployment.
• Fine-tuning and evaluation scripts for rapid prototyping.

Weights and Biases Training