ContextualAI

This repo contains the model and tokenizer checkpoints for: - model family mistralai/Mistral-7B-Instruct-v0.2 - optimized with the loss KTO - aligned using the snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset - via 3 iterations of KTO on one epoch of each training partition, each previous iteration's model serving as the reference for the subsequent. [03/06/2024]: We are #2 on the (verified) Alpaca Eval 2.0 Leaderboard scoring 33.23! To prompt this model, ensure that the format is consistent with that of TuluV2. For example, a prompt should be formatted as follows, where ` ` corresponds to the human's role and ` ` corresponds to the LLM's role. The human should speak first: Note that a beginning-of-sequence (BOS) token is automatically added at tokenization time and does not have to be added by you. No end-of-sequence (EOS) token is added to the prompt. You may also use our tokenizer's `applychattemplate` if doing inference with `chatml` set or evaluating generations through non-local clients. For more info on KTO refer to our code repository or blog for more details on the methodology. If you found this work useful, feel free to cite our work: