ytu-ce-cosmos

Turkish-Gemma-9b-T1 is based on ytu-ce-cosmos/Turkish-Gemma-9b-v0.1, adapted specifically for multi-step reasoning (“thinking”) in Turkish. The model is designed to perform better at mathematical problems, logical reasoning, step-by-step inference, and planning tasks, while still following instructions to produce clear and concise final answers. - Multi-step reasoning: Stronger intermediate inference when multiple clues/conditions are involved. - Math & logic: Improved accuracy on arithmetic, probability, sequences, rational reasoning, and logic puzzles. - Better instruction following: Better adherence to prompts. - Reduced hallucinations: The reasoning model hallucinates less, focusing on grounded answers and indicating uncertainty when necessary. To evaluate model performance, we compiled a dataset of 1,450 carefully designed questions across diverse categories. Each question was reviewed and rated by 18 human annotators, allowing for a reliable comparison across multiple models. | Model Name | Win Rate | | -------------------------------------------- | ---------- | | ytu-ce-cosmos/Turkish-Gemma-9b-T1 | 68.65% | | ytu-ce-cosmos/Turkish-Gemma-9b-T0 | 67.58% | | Qwen3-32B | 67.20% | | Qwen3-14B | 67.20% | | google/gemma-3-27b-it | 65.81% | | google/gemma-3-12b-it | 59.72% | | google/gemma-2-27b-it | 52.24% | | ytu-ce-cosmos/Turkish-Gemma-9b-v0.1 | 52.12% | | google/gemma-2-9b-it | 48.94% | A question and two answers from different models were presented to human judges. The judges selected the better answer based on their preferences. For example, in the question below, the judge evaluated both answers as good: | Model Name | Gsm8K | | --------------------------------------- | --------- | | Qwen/Qwen2.5-72B-Instruct | 83.60 | | Qwen/Qwen2.5-32B-Instruct | 77.83 | | google/gemma-3-27b-it | 77.52 | | ytu-ce-cosmos/Turkish-Gemma-9b-T1 | 77.41 | | Qwen/Qwen2.5-14B-it | 76.77 | | google/gemma-2-27b-it | 76.54 | | ytu-ce-cosmos/Turkish-Gemma-9b-v0.1 | 73.42 | | google/gemma-3-12b-it | 72.06 | | meta-llama/Llama-3-1-70B-Instruct | 66.13 | | Qwen/Qwen2.5-7B-Instruct | 64.16 | | google/gemma-2-9b-it | 63.10 | | ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1 | 59.87 | > Note: When running Turkish evaluations on well-known benchmarks, it is important to adjust the evaluation configurations specifically for reasoning models. Default settings may not reflect the true performance, as factors like context handling and prompt formatting can significantly affect results. Carefully tuning these configs ensures fairer and more accurate comparisons across models. The examples below demonstrate how to use the model to generate content based on given inputs. Tips > Use `Temperature=0.6`, `TopP=0.95`, `TopK=20`, and `MinP=0` (the default setting in `generationconfig.json`). DO NOT use greedy decoding, as it can lead to performance degradation and endless repetitions. - Complex tasks: Increase `maxnewtokens`. You can increase the `repetition penalty` and also adjust the `presencepenalty` parameter (between 0 and 2) to reduce endless repetitions. However, using a higher value may occasionally result in language mixing and a slight decrease in model performance Thanks to Hugging Face for hosting models on S3 storage. Compute resources were provided by the Barcelona Supercomputing Center COSMOS AI Research Group – Yildiz Technical University, Computer Engineering Department 🔗 https://cosmos.yildiz.edu.tr/ ✉️ [email protected]

This is a finetune version of model intfloat/multilingual-e5-large-instruct with various Turkish datasets. Recommended Instruct: "Given a Turkish search query, retrieve relevant passages written in Turkish that best answer the query" Contact COSMOS AI Research Group, Yildiz Technical University Computer Engineering Department https://cosmos.yildiz.edu.tr/ [email protected]

This is the Turkish-Gemma-9b-v0.1. This model is based on Gemma-2-9b, and was developed through a combination of continual pre-training, supervised fine-tuning (SFT), direct preference optimization (DPO), and model merging. The Turkish-Gemma-9b-v0.1 is designed for Turkish text generation tasks, providing coherent, contextually relevant continuations and answers. Due to the diverse nature of the training data—which includes large-scale pre-training corpora, instruction-tuning data, and human preference data—the model may exhibit biases. Users should be aware of these and deploy the model responsibly. You can easily demo the model here (Coming soon!): https://cosmos.yildiz.edu.tr/cosmosllm To evaluate model performance, we compiled a dataset of 1,450 carefully designed questions across diverse categories. Each question was reviewed and rated by 18 human annotators, allowing for a reliable comparison across multiple models. | Model Name | Win Rate | |---------------------------------------------|-----------------| | Qwen/Qwen3-30B-A3B | 62.39% | | gpt-4o-mini | 62.12% | | google/gemma-3-12b-it | 61.61% | | google/gemma-2-27b-it | 57.91% | | ytu-ce-cosmos/Turkish-Gemma-9b-v0.1 | 57.30% | | google/gemma-2-9b-it | 54.13% | | ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1 | 36.89% | A question and two answers from different models were presented to human judges. The judges selected the better answer based on their preferences. For example, in the question below, the judge selected the answer on the right: 📊 Turkish Evaluation Benchmark Results (via `malhajar17/lm-evaluation-harnessturkish`) | Model Name | Average | MMLU | TruthfulQA | ARC | Hellaswag | Gsm8K | Winogrande | |---------------------------------------------|---------|-------|--------------|-------|-----------|-------|------------| | Qwen/Qwen2.5-72B-Instruct | 67.69 | 77.28 | 59.86 | 61.52 | 61.98 | 83.6 | 61.92 | | google/gemma-3-27b-it | 67.36 | 70.2 | 57.06 | 66.98 | 66.58 | 77.52 | 65.8 | | google/gemma-2-27b-it | 65.57 | 66.49 | 57.45 | 63.65 | 63.86 | 76.54 | 65.4 | | meta-llama/Llama-3-1-70B-Instruct | 63.92 | 74.00 | 51.41 | 59.64 | 64.31 | 66.13 | 66.90 | | Qwen/Qwen2.5-32B-Instruct | 63.74 | 70.93 | 57.87 | 57.00 | 57.04 | 77.83 | 61.77 | | ytu-ce-cosmos/Turkish-Gemma-9b-v0.1 | 63.31 | 63.85 | 54.21 | 59.64 | 64.19 | 73.42 | 64.53 | | google/gemma-3-12b-it | 62.94 | 63.92 | 57.16 | 60.67 | 62.00 | 72.06 | 61.77 | | Qwen/Qwen2.5-14B-it | 60.34 | 65.28 | 59.00 | 50.00 | 52.22 | 76.77 | 58.77 | | google/gemma-2-9b-it | 59.14 | 61.07 | 55.77 | 56.31 | 56.48 | 63.10 | 62.09 | | ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1 | 55.03 | 51.97 | 57.56 | 51.02 | 52.96 | 59.87 | 57.77 | | Qwen/Qwen2.5-7B-Instruct | 53.42 | 56.31 | 55.99 | 42.06 | 44.71 | 64.16 | 59.66 | Acknowledgments - Thanks to the generous support from the Hugging Face team, it is possible to download models from their S3 storage 🤗 - Computing resources used in this work were provided by the National Center for High Performance Computing of Turkey (UHeM) under grant numbers 1016912023 and 1018512024 Contact COSMOS AI Research Group, Yildiz Technical University Computer Engineering Department https://cosmos.yildiz.edu.tr/ [email protected]

Objective Due to the need for quantized models in real-time applications, we introduce our GGUF formatted models. These models are part of GGML project with a hope to democratize the use of Large Models. Depending on the quantization type, there are 20+ models. Features All quantization details are listed on the right by Hugging Face. All the models have been tested in `llama.cpp` environments, `llama-cli` and `llama-server`. Furthermore, a YouTube video has been made to introduce the basics ...

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. - Developed by: [More Information Needed] - Funded by [optional]: [More Information Needed] - Shared by [optional]: [More Information Needed] - Model type: [More Information Needed] - Language(s) (NLP): [More Information Needed] - License: [More Information Needed] - Finetuned from model [optional]: [More Information Needed] - Repository: [More Information Needed] - Paper [optional]: [More Information Needed] - Demo [optional]: [More Information Needed] Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - Hardware Type: [More Information Needed] - Hours used: [More Information Needed] - Cloud Provider: [More Information Needed] - Compute Region: [More Information Needed] - Carbon Emitted: [More Information Needed]