GRMenon

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on the mentalhealthcounselingconversations dataset. It achieves the following results on the evaluation set: - Loss: 0.6432 A Mistral-7B-Instruct-v0.2 model finetuned on a corpus of mental health conversations between a psychologist and a user. The intention was to create a mental health assistant, "Connor", to address user questions based on responses from a psychologist. The model is finetuned on a corpus of mental health conversations between a psychologist and a client, in the form of context - response pairs. This dataset is a collection of questions and answers sourced from two online counseling and therapy platforms. The questions cover a wide range of mental health topics, and the answers are provided by qualified psychologists. Dataset found here :- Kaggle Huggingface The following hyperparameters were used during training: - learningrate: 0.0002 - trainbatchsize: 8 - evalbatchsize: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lrschedulertype: cosine - lrschedulerwarmupratio: 0.05 - numepochs: 3 - mixedprecisiontraining: Native AMP | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:----:|:---------------:| | 1.4325 | 1.0 | 352 | 0.9064 | | 1.2608 | 2.0 | 704 | 0.6956 | | 1.1845 | 3.0 | 1056 | 0.6432 | - PEFT 0.7.1 - Transformers 4.36.1 - Pytorch 2.0.0 - Datasets 2.1.0 - Tokenizers 0.15.0