roleplaiapp
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Q4_K_M-GGUF
Midnight-Miqu-70B-v1.5-i1-Q4_K_M-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q4_K_M-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q4_0-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_S-GGUF
Llama-3.3-70B-Instruct-Q4_K_M-GGUF
Repo: `roleplaiapp/Llama-3.3-70B-Instruct-Q4KM -GGUF` Original Model: `Llama-3.3-70B-Instruct` Organization: `meta-llama` Quantized File: `llama-3.3-70b-instruct-q3km.gguf` Quantization: `GGUF` Quantization Method: `Q4KM ` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q4KM quantized version of Llama-3.3-70B-Instruct. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-70B-Q4_0-GGUF
Llama-3.1-Nemotron-70B-Instruct-HF-Q4_K_M-GGUF
Llama-3.3-70B-Instruct-Q4_0-GGUF
DeepSeek-R1-Distill-Llama-70B-Q4_K_S-GGUF
roleplaiapp/DeepSeek-R1-Distill-Llama-70B-Q4KS-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Llama-70B-Q4KS-GGUF` Original Model: `DeepSeek-R1-Distill-Llama-70B` Organization: `deepseek-ai` Quantized File: `deepseek-r1-distill-llama-70b-q4ks.gguf` Quantization: `GGUF` Quantization Method: `Q4KS` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q4KS quantized version of DeepSeek-R1-Distill-Llama-70B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Q4_K_S-GGUF
Llama-3.1-Nemotron-70B-Instruct-HF-Q4_0-GGUF
TunnedLlama-3.1-8B_v2-Q8_0-GGUF
Repo: `roleplaiapp/TunnedLlama-3.1-8Bv2-Q80-GGUF` Original Model: `TunnedLlama-3.1-8Bv2` Quantized File: `TunnedLlama-3.1-8Bv2.Q80.gguf` Quantization: `GGUF` Quantization Method: `Q80` Overview This is a GGUF Q80 quantized version of TunnedLlama-3.1-8Bv2 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
MN-12B-Mag-Mell-R1-Q4_K_M-GGUF
Gemma The Writer N Restless Quill 10B Uncensored IQ4 XS GGUF
roleplaiapp/Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-IQ4XS-GGUF Repo: `roleplaiapp/Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-IQ4XS-GGUF` Original Model: `Gemma-The-Writer-N-Restless-Quill-10B-Uncensored` Quantized File: `Gemma-The-Writer-N-Restless-Quill-10B-DAU-IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of Gemma-The-Writer-N-Restless-Quill-10B-Uncensored Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q8_0-GGUF
Llama-3.2-3B-Instruct-uncensored-Q6_K-GGUF
Qwen2.5-7B-Instruct-Uncensored-f16-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q3_K_M-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q2_K-GGUF
DeepSeek-R1-Distill-Llama-70B-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q4_K_S-GGUF
mistral_fp8
Mistral-Small-3.2-24B-Instruct-2506 is a minor update of Mistral-Small-3.1-24B-Instruct-2503. Small-3.2 improves in the following categories: - Instruction following: Small-3.2 is better at following precise instructions - Repetition errors: Small-3.2 produces less infinite generations or repetitive answers - Function calling: Small-3.2's function calling template is more robust (see here and examples) In all other categories Small-3.2 should match or slightly improve compared to Mistral-Small-3.1-24B-Instruct-2503. Key Features - same as Mistral-Small-3.1-24B-Instruct-2503 We compare Mistral-Small-3.2-24B to Mistral-Small-3.1-24B-Instruct-2503. For more comparison against other models of similar size, please check Mistral-Small-3.1's Benchmarks' | Model | Wildbench v2 | Arena Hard v2 | IF (Internal; accuracy) | |-------|---------------|---------------|------------------------| | Small 3.1 24B Instruct | 55.6% | 19.56% | 82.75% | | Small 3.2 24B Instruct | 65.33% | 43.1% | 84.78% | Small 3.2 reduces infinite generations by 2x on challenging, long and repetitive prompts. | Model | Infinite Generations (Internal; Lower is better) | |-------|-------| | Small 3.1 24B Instruct | 2.11% | | Small 3.2 24B Instruct | 1.29% | | Model | MMLU | MMLU Pro (5-shot CoT) | MATH | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT )| MBPP Plus - Pass@5 | HumanEval Plus - Pass@5 | SimpleQA (TotalAcc)| |--------------------------------|-----------|-----------------------|------------------------|------------------------|---------------------------|--------------------|-------------------------|--------------------| | Small 3.1 24B Instruct | 80.62% | 66.76% | 69.30% | 44.42% | 45.96% | 74.63% | 88.99% | 10.43% | | Small 3.2 24B Instruct | 80.50% | 69.06% | 69.42% | 44.22% | 46.13% | 78.33% | 92.90% | 12.10% | | Model | MMMU | Mathvista | ChartQA | DocVQA | AI2D | |--------------------------------|------------|-----------|-----------|-----------|-----------| | Small 3.1 24B Instruct | 64.00% | 68.91%| 86.24% | 94.08% | 93.72% | | Small 3.2 24B Instruct | 62.50% | 67.09% | 87.4% | 94.86% | 92.91% | The model can be used with the following frameworks; - `vllm (recommended)`: See here - `transformers`: See here Note 1: We recommend using a relatively low temperature, such as `temperature=0.15`. Note 2: Make sure to add a system prompt to the model to best tailor it to your needs. If you want to use the model as a general assistant, we recommend to use the one provided in the SYSTEMPROMPT.txt file. Doing so should automatically install `mistralcommon >= 1.6.2`. You can also make use of a ready-to-go docker image or on the docker hub. We recommend that you use Mistral-Small-3.2-24B-Instruct-2506 in a server/client setting. Note: Running Mistral-Small-3.2-24B-Instruct-2506 on GPU requires ~55 GB of GPU RAM in bf16 or fp16. 2. To ping the client you can use a simple Python snippet. See the following examples. Leverage the vision capabilities of Mistral-Small-3.2-24B-Instruct-2506 to make the best choice given a scenario, go catch them all ! Mistral-Small-3.2-24B-Instruct-2506 is excellent at function / tool calling tasks via vLLM. E.g.: Mistral-Small-3.2-24B-Instruct-2506 will follow your instructions down to the last letter ! You can also use Mistral-Small-3.2-24B-Instruct-2506 with `Transformers` ! To make the best use of our model with `Transformers` make sure to have installed `mistral-common >= 1.6.2` to use our tokenizer. Then load our tokenizer along with the model and generate:
DeepSeek-R1-Distill-Llama-70B-Q3_K_M-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q4_0-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q4_K_M-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-f16-GGUF
roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-f16-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-f16-GGUF` Original Model: `DeepSeek-R1-Distill-Alpaca-FineTuned` Quantized File: `DeepSeek-R1-Distill-Alpaca-FineTuned.f16.gguf` Quantization: `GGUF` Quantization Method: `f16` Overview This is a GGUF f16 quantized version of DeepSeek-R1-Distill-Alpaca-FineTuned Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q3_K_L-GGUF
Pathfinder-RP-12B-RU-Q6_K-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q4_K_S-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q5_0-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-i1-Q4_K_M-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q3_K_M-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q6_K-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q5_K_S-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-IQ4_XS-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q4_0-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-i1-IQ3_M-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q4_K_M-GGUF
roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q4KM-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q4KM-GGUF` Original Model: `DeepSeek-R1-Distill-Alpaca-FineTuned` Quantized File: `DeepSeek-R1-Distill-Alpaca-FineTuned.Q4KM.gguf` Quantization: `GGUF` Quantization Method: `Q4KM` Overview This is a GGUF Q4KM quantized version of DeepSeek-R1-Distill-Alpaca-FineTuned Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24B-gguf-Q8_0-GGUF
oh-dcft-v3.1-claude-3-5-haiku-20241022-Q3_K_L-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q5_K_M-GGUF
roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q5KM-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q5KM-GGUF` Original Model: `DeepSeek-R1-Distill-Alpaca-FineTuned` Quantized File: `DeepSeek-R1-Distill-Alpaca-FineTuned.Q5KM.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Overview This is a GGUF Q5KM quantized version of DeepSeek-R1-Distill-Alpaca-FineTuned Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-70B-Q6_K-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q3_K_M-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-IQ4_XS-GGUF
roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-IQ4XS-GGUF Repo: `roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-IQ4XS-GGUF` Original Model: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B` Quantized File: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B.IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-70B-Q3_K_S-GGUF
MistralRP-Noromaid-NSFW-Mistral-7B-Q8_0-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q6_K-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-i1-Q2_K-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q3_K_L-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q3_K_M-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q5_K_M-GGUF
DeepSeek-R1-Distill-Llama-8B-Q4_0-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q4_K_M-GGUF
Janus-Pro-7B-LM-Q8_0-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-i1-Q3_K_M-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q3_K_S-GGUF
roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q3KS-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q3KS-GGUF` Original Model: `DeepSeek-R1-Distill-Alpaca-FineTuned` Quantized File: `DeepSeek-R1-Distill-Alpaca-FineTuned.Q3KS.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Overview This is a GGUF Q3KS quantized version of DeepSeek-R1-Distill-Alpaca-FineTuned Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q6_K-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q3_K_S-GGUF
roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q3KS-GGUF Repo: `roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q3KS-GGUF` Original Model: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B` Quantized File: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B.Q3KS.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Overview This is a GGUF Q3KS quantized version of DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
MistralRP-Noromaid-NSFW-Mistral-7B-Q5_K_M-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-IQ3_S-GGUF
Janus-Pro-7B-LM-Q4_K_M-GGUF
AceInstruct-1.5B-Q4_K_S-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q3_K_L-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-f16-GGUF
roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-f16-GGUF Repo: `roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-f16-GGUF` Original Model: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B` Quantized File: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B.f16.gguf` Quantization: `GGUF` Quantization Method: `f16` Overview This is a GGUF f16 quantized version of DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Qwen-32B-Q4_K_M-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q2_K-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q3_K_S-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-IQ3_M-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q4_K_S-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q4_K_S-GGUF
roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q4KS-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q4KS-GGUF` Original Model: `DeepSeek-R1-Distill-Alpaca-FineTuned` Quantized File: `DeepSeek-R1-Distill-Alpaca-FineTuned.Q4KS.gguf` Quantization: `GGUF` Quantization Method: `Q4KS` Overview This is a GGUF Q4KS quantized version of DeepSeek-R1-Distill-Alpaca-FineTuned Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24B-gguf-IQ4_XS-GGUF
roleplaiapp/Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24B-gguf-IQ4XS-GGUF Repo: `roleplaiapp/Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24B-gguf-IQ4XS-GGUF` Original Model: `Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24B-gguf` Quantized File: `M-MOE-4X7B-Dark-MultiVerse-UC-E32-24B-DAU-IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24B-gguf Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
saiga_nemo_12b_gguf-Q8_0-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q4_K_M-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q3_K_S-GGUF
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-IQ3_XS-GGUF
Codestral-22B-v0.1-Q4_K_M-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q8_0-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q8_0-GGUF
roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q80-GGUF Repo: `roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q80-GGUF` Original Model: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B` Quantized File: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B.Q80.gguf` Quantization: `GGUF` Quantization Method: `Q80` Overview This is a GGUF Q80 quantized version of DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Alpaca-FineTuned-Q5_K_S-GGUF
roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q5KS-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-Q5KS-GGUF` Original Model: `DeepSeek-R1-Distill-Alpaca-FineTuned` Quantized File: `DeepSeek-R1-Distill-Alpaca-FineTuned.Q5KS.gguf` Quantization: `GGUF` Quantization Method: `Q5KS` Overview This is a GGUF Q5KS quantized version of DeepSeek-R1-Distill-Alpaca-FineTuned Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
AceInstruct-1.5B-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q2_K-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q6_K-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-IQ4_XS-GGUF
roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-IQ4XS-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Alpaca-FineTuned-IQ4XS-GGUF` Original Model: `DeepSeek-R1-Distill-Alpaca-FineTuned` Quantized File: `DeepSeek-R1-Distill-Alpaca-FineTuned.IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of DeepSeek-R1-Distill-Alpaca-FineTuned Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-3.1-16.5B-Brainstorm-i1-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q4_0-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q3_K_M-GGUF
roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q3KM-GGUF Repo: `roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q3KM-GGUF` Original Model: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B` Quantized File: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B.Q3KM.gguf` Quantization: `GGUF` Quantization Method: `Q3KM` Overview This is a GGUF Q3KM quantized version of DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Alpaca-FineTuned-Q8_0-GGUF
Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q8_0-GGUF
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-i1-Q4_K_S-GGUF
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-IQ4_XS-GGUF
DeepSeek-R1-Distill-Llama-8B-Q4_K_M-GGUF
DeepSeek-R1-Distill-Llama-8B-Q3_K_S-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q6_K-GGUF
DS-R1-Distill-Q2.5-14B-Harmony_V0.1-Q3_K_M-GGUF
roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q3KM-GGUF Repo: `roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q3KM-GGUF` Original Model: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1` Quantized File: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1.Q3KM.gguf` Quantization: `GGUF` Quantization Method: `Q3KM` Overview This is a GGUF Q3KM quantized version of DS-R1-Distill-Q2.5-14B-HarmonyV0.1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
AceInstruct-1.5B-Q8_0-GGUF
ALIA-40b-Q2_K-GGUF
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q5_K_M-GGUF
Qwen2.5-Coder-14B-Instruct-Uncensored-Q4_K_S-GGUF
Llama-3-monika-ddlc-11.5b-v1-i1-IQ3_XS-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q6_K-GGUF
Omni-Reasoner-2B-Q3_K_S-GGUF
Repo: `roleplaiapp/Omni-Reasoner-2B-Q3KS-GGUF` Original Model: `Omni-Reasoner-o1` Organization: `prithivMLmods` Quantized File: `omni-reasoner-2b-q3ks.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q3KS quantized version of Omni-Reasoner-o1. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Qwen-1.5B-Q6_K-GGUF
Llama-3.2-3B-Instruct-uncensored-Q2_K-GGUF
MN-12B-Mag-Mell-R1-Q8_0-GGUF
MN-12B-Mag-Mell-R1-Q5_K_M-GGUF
14B-Qwen2.5-Kunou-v1-Q3_K_L-GGUF
Dolphin3.0-Llama3.1-8B-Q4_K_S-GGUF
Omni-Reasoner-2B-Q4_K_S-GGUF
Repo: `roleplaiapp/Omni-Reasoner-2B-Q4KS-GGUF` Original Model: `Omni-Reasoner-o1` Organization: `prithivMLmods` Quantized File: `omni-reasoner-2b-q4ks.gguf` Quantization: `GGUF` Quantization Method: `Q4KS` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q4KS quantized version of Omni-Reasoner-o1. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Omni-Reasoner-2B-Q5_K_M-GGUF
Repo: `roleplaiapp/Omni-Reasoner-2B-Q5KM-GGUF` Original Model: `Omni-Reasoner-o1` Organization: `prithivMLmods` Quantized File: `omni-reasoner-2b-q5km.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q5KM quantized version of Omni-Reasoner-o1. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-8B-Q5_0-GGUF
Virtuoso-Lite-Q5_K_S-GGUF
Mistral-Small-24B-Instruct-2501-Q2_K-GGUF
Reasoning-Llama-3.1-CoT-RE1-f16-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q5_K_M-GGUF
ReaderLM-v2-Q2_K-GGUF
Codestral-22B-v0.1-Q5_K_M-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q3_K_S-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q3_K_L-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q4_K_S-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q6_K-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q3_K_S-GGUF
oh-dcft-v3.1-claude-3-5-haiku-20241022-Q2_K-GGUF
phi-4-Q3_K_S-GGUF
SmallThinker-3B-Preview-Q4_0-GGUF
Codestral-22B-v0.1-Q6_K-GGUF
Omni-Reasoner-2B-Q4_0-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q5_K_M-GGUF
SILMA-Kashif-2B-Instruct-v1.0-i1-Q3_K_S-GGUF
oh-dcft-v3.1-claude-3-5-haiku-20241022-Q8_0-GGUF
Minerva-14b-V0.1-i1-IQ4_XS-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q6_K-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q6K-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q6K-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q6K.gguf` Quantization: `GGUF` Quantization Method: `Q6K` Overview This is a GGUF Q6K quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-Q4_K_M-GGUF
Llama3-Chinese-8B-Instruct-Q5_K_S-GGUF
Llama-3.3-70B-Instruct-Q3_K_S-GGUF
DS-R1-Distill-Q2.5-14B-Harmony_V0.1-Q6_K-GGUF
roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q6K-GGUF Repo: `roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q6K-GGUF` Original Model: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1` Quantized File: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1.Q6K.gguf` Quantization: `GGUF` Quantization Method: `Q6K` Overview This is a GGUF Q6K quantized version of DS-R1-Distill-Q2.5-14B-HarmonyV0.1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Minerva-14b-V0.1-i1-Q6_K-GGUF
Jaja-small-v1-Q8_0-GGUF
Repo: `roleplaiapp/Jaja-small-v1-Q80-GGUF` Original Model: `Jaja-small-v1` Quantized File: `Jaja-small-v1.Q80.gguf` Quantization: `GGUF` Quantization Method: `Q80` Overview This is a GGUF Q80 quantized version of Jaja-small-v1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Qwen2.5-Coder-14B-Instruct-Uncensored-IQ4_XS-GGUF
roleplaiapp/Qwen2.5-Coder-14B-Instruct-Uncensored-IQ4XS-GGUF Repo: `roleplaiapp/Qwen2.5-Coder-14B-Instruct-Uncensored-IQ4XS-GGUF` Original Model: `Qwen2.5-Coder-14B-Instruct-Uncensored` Quantized File: `Qwen2.5-Coder-14B-Instruct-Uncensored.IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of Qwen2.5-Coder-14B-Instruct-Uncensored Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-8B-Q6_K-GGUF
Llama-3.3-70B-Instruct-Q5_0-GGUF
AceInstruct-72B-Q5_K_M-GGUF
14B-Qwen2.5-Kunou-v1-IQ4_XS-GGUF
DS-Distilled-Hermes-Llama-3.1_TIES-i1-Q4_K_M-GGUF
Mistral-Small-24B-Instruct-2501-IQ3_XS-GGUF
roleplaiapp/Mistral-Small-24B-Instruct-2501-IQ3XS-GGUF Repo: `roleplaiapp/Mistral-Small-24B-Instruct-2501-IQ3XS-GGUF` Original Model: `Mistral-Small-24B-Instruct-2501` Quantized File: `Mistral-Small-24B-Instruct-2501-IQ3XS.gguf` Quantization: `GGUF` Quantization Method: `IQ3XS` Overview This is a GGUF IQ3XS quantized version of Mistral-Small-24B-Instruct-2501 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Llama-3.1-Nemotron-70B-Instruct-HF-Q3_K_S-GGUF
roleplaiapp/Llama-3.1-Nemotron-70B-Instruct-HF-Q3KS-GGUF Repo: `roleplaiapp/Llama-3.1-Nemotron-70B-Instruct-HF-Q3KS-GGUF` Original Model: `Llama-3.1-Nemotron-70B-Instruct-HF` Organization: `nvidia` Quantized File: `llama-3.1-nemotron-70b-instruct-hf-q3ks.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q3KS quantized version of Llama-3.1-Nemotron-70B-Instruct-HF. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Wayfarer-12B-Q5_K_M-GGUF
Llama-3.1-Nemotron-70B-Instruct-HF-Q5_K_S-GGUF
Qwen2.5-7B-Instruct-Uncensored-IQ4_XS-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q4_K_S-GGUF
DS-R1-Distill-Q2.5-14B-Harmony_V0.1-Q3_K_S-GGUF
roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q3KS-GGUF Repo: `roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q3KS-GGUF` Original Model: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1` Quantized File: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1.Q3KS.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Overview This is a GGUF Q3KS quantized version of DS-R1-Distill-Q2.5-14B-HarmonyV0.1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
deepseek-r1-qwen-2.5-32B-ablated-IQ4_XS-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-Q4_K_M-GGUF
saiga_nemo_12b_gguf-Q5_K_M-GGUF
Confucius-o1-14B-Q6_K-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q8_0-GGUF
ALIA-40b-Q4_0-GGUF
Llama-3.3-70B-Instruct-Q2_K-GGUF
Llama-3.3-70B-Instruct-Q6_K-GGUF
SmallThinker-3B-Preview-Q5_0-GGUF
internlm3-8b-instruct-Q5_0-GGUF
Wayfarer-12B-Q2_K-GGUF
AceInstruct-1.5B-Q3_K_S-GGUF
DeepSeek-R1-Distill-Llama-8B-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q5_K_M-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q5_K_M-GGUF
Midnight-Miqu-70B-v1.5-i1-Q3_K_L-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q3_K_M-GGUF
DS-Distilled-Hermes-Llama-3.1_TIES-i1-Q6_K-GGUF
DS-R1-Distill-Q2.5-14B-Harmony_V0.1-Q5_K_S-GGUF
roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q5KS-GGUF Repo: `roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q5KS-GGUF` Original Model: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1` Quantized File: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1.Q5KS.gguf` Quantization: `GGUF` Quantization Method: `Q5KS` Overview This is a GGUF Q5KS quantized version of DS-R1-Distill-Q2.5-14B-HarmonyV0.1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Q2_K-GGUF
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K-GGUF
roleplaiapp/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2K-GGUF Repo: `roleplaiapp/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2K-GGUF` Original Model: `FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview` Quantized File: `FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-Q5_K_M-GGUF
roleplaiapp/DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-Q5KM-GGUF Repo: `roleplaiapp/DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-Q5KM-GGUF` Original Model: `DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf` Quantized File: `DeepSeek-R1-Distill-Qwen-32B-Japanese-Q5KM.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Overview This is a GGUF Q5KM quantized version of DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
internlm3-8b-instruct-Q4_K_M-GGUF
ReaderLM-v2-Q4_K_M-GGUF
Llama-3.3-70B-Instruct-Q4_K_S-GGUF
Llama-3.3-70B-Instruct-Q8_0-GGUF
SmallThinker-3B-Preview-IQ4_NL-GGUF
QwQ-32B-Preview-Q5_0-GGUF
Repo: `roleplaiapp/QwQ-32B-Preview-Q50-GGUF` Original Model: `QwQ-32B-Preview` Organization: `Qwen` Quantized File: `qwq-32b-preview-q50.gguf` Quantization: `GGUF` Quantization Method: `Q50` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q50 quantized version of QwQ-32B-Preview. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
QwQ-32B-Preview-Q4_0-GGUF
AceInstruct-72B-Q4_0-GGUF
Codestral-22B-v0.1-Q5_0-GGUF
AceInstruct-1.5B-Q4_0-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-14B-Q5_K_S-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q5_0-GGUF
DeepSeek-R1-Distill-Llama-70B-Q5_0-GGUF
14B-Qwen2.5-Kunou-v1-Q5_K_S-GGUF
Chocolatine-2-14B-Instruct-v2.0b2-i1-Q4_K_S-GGUF
Llama-3-monika-ddlc-11.5b-v1-i1-Q4_K_S-GGUF
Qwen2.5-32B-DeepSeek-R1-Instruct-i1-Q6_K-GGUF
AceInstruct-72B-Q4_K_M-GGUF
Repo: `roleplaiapp/AceInstruct-72B-Q4KM-GGUF` Original Model: `AceInstruct-72B` Organization: `nvidia` Quantized File: `aceinstruct-72b-q4km.gguf` Quantization: `GGUF` Quantization Method: `Q4KM` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q4KM quantized version of AceInstruct-72B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
SmallThinker-3B-Preview-Q4_K_S-GGUF
SmallThinker-3B-Preview-IQ3_M-GGUF
Llama-3.1-Nemotron-70B-Instruct-HF-Q5_0-GGUF
DeepSeek-R1-Distill-Llama-70B-Q8_0-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q4_K_M-GGUF
DS-Distilled-Hermes-Llama-3.1_TIES-i1-Q3_K_S-GGUF
Virtuoso-Lite-Q4_K_M-GGUF
Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q3_K_M-GGUF
Qwen2.5-Coder-14B-Instruct-Uncensored-Q5_K_M-GGUF
Qwen2.5-7B-olm-v1.4-i1-Q2_K-GGUF
Repo: `roleplaiapp/Qwen2.5-7B-olm-v1.4-i1-Q2K-GGUF` Original Model: `Qwen2.5-7B-olm-v1.4-i1` Quantized File: `Qwen2.5-7B-olm-v1.4.i1-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of Qwen2.5-7B-olm-v1.4-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
ArxivLlama-3.1-8B-Q3_K_L-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q3_K_L-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q3KL-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q3KL-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q3KL.gguf` Quantization: `GGUF` Quantization Method: `Q3KL` Overview This is a GGUF Q3KL quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Qwen2.5-32B-DeepSeek-R1-Instruct-i1-Q5_K_M-GGUF
Janus-Pro-7B-LM-f16-GGUF
QwQ-32B-Preview-Q8_0-GGUF
internlm3-8b-instruct-Q4_0-GGUF
ReaderLM-v2-Q5_K_M-GGUF
AceInstruct-72B-Q3_K_S-GGUF
Codestral-22B-v0.1-Q5_K_S-GGUF
Llama-3.1-Nemotron-70B-Instruct-HF-Q5_K_M-GGUF
DeepSeek-R1-Distill-Llama-8B-Q3_K_M-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_S-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q5_K_S-GGUF
14B-Qwen2.5-Kunou-v1-Q4_K_M-GGUF
Qwen2.5-7B-olm-v1.4-i1-IQ3_XS-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-IQ3_M-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-IQ3_XS-GGUF
SmallThinker-3B-Preview-Q3_K_S-GGUF
SmallThinker-3B-Preview-Q4_K_M-GGUF
Qwen2.5-Coder-14B-Instruct-Uncensored-Q4_K_M-GGUF
AceInstruct-72B-Q2_K-GGUF
Codestral-22B-v0.1-Q4_0-GGUF
DeepSeek-R1-Distill-Llama-8B-Q8_0-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_L-GGUF
14B-Qwen2.5-Kunou-v1-Q3_K_M-GGUF
GRAG-R1-14B-SFT-DE-EXP-Q6_K-GGUF
Slush-Sunfall-Rocinante-GGLD-12B-Q3_K_L-GGUF
Slush-Sunfall-Rocinante-GGLD-12B-Q5_K_S-GGUF
deepseek-r1-qwen-2.5-32B-ablated-IQ3_M-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-IQ3M-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-IQ3M-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-IQ3M.gguf` Quantization: `GGUF` Quantization Method: `IQ3M` Overview This is a GGUF IQ3M quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
deepseek-r1-qwen-2.5-32B-ablated-IQ3_XS-GGUF
RuadaptQwen2.5-32B-Pro-Beta-Q2_K-GGUF
Repo: `roleplaiapp/RuadaptQwen2.5-32B-Pro-Beta-Q2K-GGUF` Original Model: `RuadaptQwen2.5-32B-Pro-Beta` Quantized File: `Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of RuadaptQwen2.5-32B-Pro-Beta Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Cydonia-24B-v2a-Q2_K-GGUF
Repo: `roleplaiapp/Cydonia-24B-v2a-Q2K-GGUF` Original Model: `Cydonia-24B-v2a` Quantized File: `Cydonia-24B-v2a-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of Cydonia-24B-v2a Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Dolphin3.0-Llama3.1-8B-Q6_K-GGUF
AceInstruct-7B-Q2_K-GGUF
Qwen2.5-32B-DeepSeek-R1-Instruct-i1-IQ4_XS-GGUF
roleplaiapp/Qwen2.5-32B-DeepSeek-R1-Instruct-i1-IQ4XS-GGUF Repo: `roleplaiapp/Qwen2.5-32B-DeepSeek-R1-Instruct-i1-IQ4XS-GGUF` Original Model: `Qwen2.5-32B-DeepSeek-R1-Instruct-i1` Quantized File: `Qwen2.5-32B-DeepSeek-R1-Instruct.i1-IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of Qwen2.5-32B-DeepSeek-R1-Instruct-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
SmallThinker-3B-Preview-Q6_K-GGUF
QwQ-32B-Preview-Q5_K_M-GGUF
internlm3-8b-instruct-Q4_K_S-GGUF
ReaderLM-v2-Q3_K_M-GGUF
AceInstruct-7B-Q4_0-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q5_K_S-GGUF
ALIA-40b-Q5_0-GGUF
Midnight-Miqu-70B-v1.5-i1-IQ3_S-GGUF
SILMA-Kashif-2B-Instruct-v1.0-i1-IQ3_M-GGUF
14B-Qwen2.5-Kunou-v1-Q3_K_S-GGUF
DS-R1-Distill-Q2.5-14B-Harmony_V0.1-Q5_K_M-GGUF
roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q5KM-GGUF Repo: `roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q5KM-GGUF` Original Model: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1` Quantized File: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1.Q5KM.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Overview This is a GGUF Q5KM quantized version of DS-R1-Distill-Q2.5-14B-HarmonyV0.1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DS-R1-Distill-Q2.5-14B-Harmony_V0.1-IQ4_XS-GGUF
Llama3.2-doker-Q4_K_M-GGUF
Jaja-small-v1-Q4_K_M-GGUF
Repo: `roleplaiapp/Jaja-small-v1-Q4KM-GGUF` Original Model: `Jaja-small-v1` Quantized File: `Jaja-small-v1.Q4KM.gguf` Quantization: `GGUF` Quantization Method: `Q4KM` Overview This is a GGUF Q4KM quantized version of Jaja-small-v1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Jaja-small-v1-f16-GGUF
TunnedLlama-3.1-8B_v2-f16-GGUF
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-f16-GGUF
Qwen2.5-32b-Erudite-Writer-i1-Q3_K_L-GGUF
roleplaiapp/Qwen2.5-32b-Erudite-Writer-i1-Q3KL-GGUF Repo: `roleplaiapp/Qwen2.5-32b-Erudite-Writer-i1-Q3KL-GGUF` Original Model: `Qwen2.5-32b-Erudite-Writer-i1` Quantized File: `Qwen2.5-32b-Erudite-Writer.i1-Q3KL.gguf` Quantization: `GGUF` Quantization Method: `Q3KL` Overview This is a GGUF Q3KL quantized version of Qwen2.5-32b-Erudite-Writer-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
medicine-LLM-Q3_K_L-GGUF
Dolphin3.0-Llama3.1-8B-Q8_0-GGUF
Dolphin3.0-Llama3.1-8B-Q4_K_M-GGUF
Wayfarer-12B-Q4_K_M-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q4_K_S-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q4_K_S-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q4KS-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q4KS-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q4KS.gguf` Quantization: `GGUF` Quantization Method: `Q4KS` Overview This is a GGUF Q4KS quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
QwQ-32B-Preview-Q6_K-GGUF
internlm3-8b-instruct-Q3_K_S-GGUF
ReaderLM-v2-Q6_K-GGUF
Wayfarer-12B-Q4_K_S-GGUF
AceInstruct-1.5B-Q3_K_M-GGUF
AceInstruct-1.5B-Q5_K_S-GGUF
Omni-Reasoner-2B-Q2_K-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q3_K_L-GGUF
DeepSeek-R1-Distill-Llama-70B-Q3_K_L-GGUF
MN-12B-Mag-Mell-R1-Q3_K_S-GGUF
Midnight-Miqu-70B-v1.5-i1-Q2_K-GGUF
Repo: `roleplaiapp/Midnight-Miqu-70B-v1.5-i1-Q2K-GGUF` Original Model: `Midnight-Miqu-70B-v1.5-i1` Quantized File: `Midnight-Miqu-70B-v1.5.i1-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of Midnight-Miqu-70B-v1.5-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DS-Distilled-Hermes-Llama-3.1_TIES-i1-Q3_K_M-GGUF
Llama3.2-doker-Q4_K_S-GGUF
Jaja-small-v1-Q6_K-GGUF
Repo: `roleplaiapp/Jaja-small-v1-Q6K-GGUF` Original Model: `Jaja-small-v1` Quantized File: `Jaja-small-v1.Q6K.gguf` Quantization: `GGUF` Quantization Method: `Q6K` Overview This is a GGUF Q6K quantized version of Jaja-small-v1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-Small-24B-Instruct-2501-Q8_0-GGUF
TunnedLlama-3.1-8B_v2-Q2_K-GGUF
Repo: `roleplaiapp/TunnedLlama-3.1-8Bv2-Q2K-GGUF` Original Model: `TunnedLlama-3.1-8Bv2` Quantized File: `TunnedLlama-3.1-8Bv2.Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of TunnedLlama-3.1-8Bv2 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Qwen2.5-7B-olm-v1.4-i1-Q3_K_S-GGUF
Repo: `roleplaiapp/Qwen2.5-7B-olm-v1.4-i1-Q3KS-GGUF` Original Model: `Qwen2.5-7B-olm-v1.4-i1` Quantized File: `Qwen2.5-7B-olm-v1.4.i1-Q3KS.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Overview This is a GGUF Q3KS quantized version of Qwen2.5-7B-olm-v1.4-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Q3_K_L-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q3_K_M-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q3KM-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q3KM-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q3KM.gguf` Quantization: `GGUF` Quantization Method: `Q3KM` Overview This is a GGUF Q3KM quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
deepseek-r1-qwen-2.5-32B-ablated-f16-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-Q4_K_S-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q4_K_S-GGUF
L3.3-Nevoria-R1-70b-IQ3_M-GGUF
Repo: `roleplaiapp/L3.3-Nevoria-R1-70b-IQ3M-GGUF` Original Model: `L3.3-Nevoria-R1-70b` Quantized File: `L3.3-Nevoria-R1-70b-IQ3M.gguf` Quantization: `GGUF` Quantization Method: `IQ3M` Overview This is a GGUF IQ3M quantized version of L3.3-Nevoria-R1-70b Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Dolphin3.0-Llama3.1-8B-Q2_K-GGUF
Dolphin3.0-Llama3.1-8B-Q5_K_S-GGUF
Crumb-13B-Q3_K_M-GGUF
Llama-3.3-70B-Instruct-Q3_K_M-GGUF
SmallThinker-3B-Preview-Q5_K_M-GGUF
SmallThinker-3B-Preview-IQ4_XS-GGUF
AceInstruct-1.5B-Q6_K-GGUF
Repo: `roleplaiapp/AceInstruct-1.5B-Q6K-GGUF` Original Model: `AceInstruct-1.5B` Organization: `nvidia` Quantized File: `aceinstruct-1.5b-q6k.gguf` Quantization: `GGUF` Quantization Method: `Q6K` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q6K quantized version of AceInstruct-1.5B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
14B-Qwen2.5-Kunou-v1-Q8_0-GGUF
Virtuoso-Lite-Q5_K_M-GGUF
GRAG-R1-14B-SFT-DE-EXP-Q4_K_M-GGUF
GRAG-R1-14B-SFT-DE-EXP-IQ4_XS-GGUF
DialoGPT-medium-anand-Q4_K_M-GGUF
Mistral-Small-24B-Instruct-2501-Q4_K_M-GGUF
roleplaiapp/Mistral-Small-24B-Instruct-2501-Q4KM-GGUF Repo: `roleplaiapp/Mistral-Small-24B-Instruct-2501-Q4KM-GGUF` Original Model: `Mistral-Small-24B-Instruct-2501` Quantized File: `Mistral-Small-24B-Instruct-2501-Q4KM.gguf` Quantization: `GGUF` Quantization Method: `Q4KM` Overview This is a GGUF Q4KM quantized version of Mistral-Small-24B-Instruct-2501 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-Small-24B-Instruct-2501-Q4_K_S-GGUF
JapMed-SLERP-IQ4_XS-GGUF
Qwen2.5-7B-olm-v1.4-i1-IQ3_M-GGUF
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Q3_K_M-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q5_K_S-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q5KS-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q5KS-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q5KS.gguf` Quantization: `GGUF` Quantization Method: `Q5KS` Overview This is a GGUF Q5KS quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
medicine-LLM-Q3_K_S-GGUF
Pantheon-RP-Pure-1.6.2-22b-Small-Q4_K_S-GGUF
roleplai-13b-v1-uncensored
ALIA-40b-Q8_0-GGUF
Dria-Agent-a-3B-Q4_K_M-GGUF
SmallThinker-3B-Preview-IQ3_XXS-GGUF
QwQ-32B-Preview-Q3_K_S-GGUF
ReaderLM-v2-Q4_0-GGUF
Dria-Agent-a-3B-Q8_0-GGUF
AceInstruct-72B-Q3_K_L-GGUF
Wayfarer-12B-Q4_0-GGUF
Wayfarer-12B-Q5_0-GGUF
Llama-3.1-Nemotron-70B-Instruct-HF-Q3_K_M-GGUF
AceInstruct-1.5B-Q5_K_M-GGUF
Repo: `roleplaiapp/AceInstruct-1.5B-Q5KM-GGUF` Original Model: `AceInstruct-1.5B` Organization: `nvidia` Quantized File: `aceinstruct-1.5b-q5km.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q5KM quantized version of AceInstruct-1.5B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Omni-Reasoner-2B-Q6_K-GGUF
Omni-Reasoner-2B-Q5_0-GGUF
Repo: `roleplaiapp/Omni-Reasoner-2B-Q50-GGUF` Original Model: `Omni-Reasoner-o1` Organization: `prithivMLmods` Quantized File: `omni-reasoner-2b-q50.gguf` Quantization: `GGUF` Quantization Method: `Q50` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q50 quantized version of Omni-Reasoner-o1. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSeek-R1-Distill-Llama-8B-Q3_K_L-GGUF
DeepSeek-R1-Distill-Llama-8B-Q5_K_S-GGUF
Midnight-Miqu-70B-v1.5-i1-Q4_K_S-GGUF
DeepSeek-R1-Distill-Qwen-32B-Uncensored-Q3_K_L-GGUF
Phi-4-ReasoningRP-Q3_K_L-GGUF
Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q4_K_M-GGUF
roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q4KM-GGUF Repo: `roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q4KM-GGUF` Original Model: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored` Quantized File: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored.Q4KM.gguf` Quantization: `GGUF` Quantization Method: `Q4KM` Overview This is a GGUF Q4KM quantized version of Qwen2.5-14B-DeepSeek-R1-1M-Uncensored Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-IQ4_XS-GGUF
roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-IQ4XS-GGUF Repo: `roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-IQ4XS-GGUF` Original Model: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored` Quantized File: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored.IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of Qwen2.5-14B-DeepSeek-R1-1M-Uncensored Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
GRAG-R1-14B-SFT-DE-EXP-Q3_K_L-GGUF
Jaja-small-v1-Q5_K_M-GGUF
Repo: `roleplaiapp/Jaja-small-v1-Q5KM-GGUF` Original Model: `Jaja-small-v1` Quantized File: `Jaja-small-v1.Q5KM.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Overview This is a GGUF Q5KM quantized version of Jaja-small-v1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Jaja-small-v1-Q5_K_S-GGUF
Repo: `roleplaiapp/Jaja-small-v1-Q5KS-GGUF` Original Model: `Jaja-small-v1` Quantized File: `Jaja-small-v1.Q5KS.gguf` Quantization: `GGUF` Quantization Method: `Q5KS` Overview This is a GGUF Q5KS quantized version of Jaja-small-v1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-Small-24B-Instruct-2501-IQ3_M-GGUF
roleplaiapp/Mistral-Small-24B-Instruct-2501-IQ3M-GGUF Repo: `roleplaiapp/Mistral-Small-24B-Instruct-2501-IQ3M-GGUF` Original Model: `Mistral-Small-24B-Instruct-2501` Quantized File: `Mistral-Small-24B-Instruct-2501-IQ3M.gguf` Quantization: `GGUF` Quantization Method: `IQ3M` Overview This is a GGUF IQ3M quantized version of Mistral-Small-24B-Instruct-2501 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-Small-24B-Instruct-2501-IQ4_XS-GGUF
roleplaiapp/Mistral-Small-24B-Instruct-2501-IQ4XS-GGUF Repo: `roleplaiapp/Mistral-Small-24B-Instruct-2501-IQ4XS-GGUF` Original Model: `Mistral-Small-24B-Instruct-2501` Quantized File: `Mistral-Small-24B-Instruct-2501-IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of Mistral-Small-24B-Instruct-2501 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Reasoning-Llama-3.1-CoT-RE1-IQ4_XS-GGUF
ArxivLlama-3.1-8B-Q3_K_M-GGUF
ArxivLlama-3.1-8B-IQ4_XS-GGUF
DeepSeek-R1-Distill-Alpaca-FineTuned-Q2_K-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q3_K_S-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q3KS-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q3KS-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q3KS.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Overview This is a GGUF Q3KS quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Qwen2.5-7B-Instruct-1M-Q4_K_S-GGUF
Qwen2.5-14B-DeepSeek-R1-1M-Q2_K-GGUF
Llama-3-monika-ddlc-11.5b-v1-i1-Q2_K-GGUF
roleplaiapp/Llama-3-monika-ddlc-11.5b-v1-i1-Q2K-GGUF Repo: `roleplaiapp/Llama-3-monika-ddlc-11.5b-v1-i1-Q2K-GGUF` Original Model: `Llama-3-monika-ddlc-11.5b-v1-i1` Quantized File: `Llama-3-monika-ddlc-11.5b-v1.i1-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of Llama-3-monika-ddlc-11.5b-v1-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
YuE-s1-7B-anneal-en-cot-Q3_K_M-GGUF
L3.3-Nevoria-R1-70b-Q4_K_M-GGUF
Pantheon-RP-Pure-1.6.2-22b-Small-Q5_K_M-GGUF
Cydonia-24B-v2a-Q3_K_M-GGUF
Cydonia-24B-v2a-Q4_K_M-GGUF
Dolphin3.0-Llama3.1-8B-Q3_K_S-GGUF
Confucius-o1-14B-f16-GGUF
plato-9b-Q5_K_M-GGUF
AceInstruct-1.5B-Q4_K_M-GGUF
Mistral-Small-24B-Instruct-2501-Q3_K_M-GGUF
roleplaiapp/Mistral-Small-24B-Instruct-2501-Q3KM-GGUF Repo: `roleplaiapp/Mistral-Small-24B-Instruct-2501-Q3KM-GGUF` Original Model: `Mistral-Small-24B-Instruct-2501` Quantized File: `Mistral-Small-24B-Instruct-2501-Q3KM.gguf` Quantization: `GGUF` Quantization Method: `Q3KM` Overview This is a GGUF Q3KM quantized version of Mistral-Small-24B-Instruct-2501 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
phi-4-Q3_K_M-GGUF
Llama-3.3-70B-Instruct-Q3_K_L-GGUF
Llama-3.3-70B-Instruct-Q5_K_S-GGUF
Llama-3.3-70B-Instruct-Q5_K_M-GGUF
Repo: `roleplaiapp/Llama-3.3-70B-Instruct-Q5KM-GGUF` Original Model: `Llama-3.3-70B-Instruct` Organization: `meta-llama` Quantized File: `llama-3.3-70b-instruct-q5km.gguf` Quantization: `GGUF` Quantization Method: `Q4KM ` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q5KM quantized version of Llama-3.3-70B-Instruct. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
AceInstruct-72B-Q3_K_M-GGUF
Codestral-22B-v0.1-Q3_K_S-GGUF
AceInstruct-7B-Q3_K_M-GGUF
AceInstruct-7B-Q5_0-GGUF
Repo: `roleplaiapp/AceInstruct-7B-Q50-GGUF` Original Model: `AceInstruct-7B` Organization: `nvidia` Quantized File: `aceinstruct-7b-q50.gguf` Quantization: `GGUF` Quantization Method: `Q50` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q50 quantized version of AceInstruct-7B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Omni-Reasoner-2B-Q8_0-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_M-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q5_0-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q5_K_S-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q3_K_M-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q6_K-GGUF
QwQ-32B-Preview-abliterated-Q3_K_L-GGUF
SILMA-Kashif-2B-Instruct-v1.0-i1-Q2_K-GGUF
roleplaiapp/SILMA-Kashif-2B-Instruct-v1.0-i1-Q2K-GGUF Repo: `roleplaiapp/SILMA-Kashif-2B-Instruct-v1.0-i1-Q2K-GGUF` Original Model: `SILMA-Kashif-2B-Instruct-v1.0-i1` Quantized File: `SILMA-Kashif-2B-Instruct-v1.0.i1-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of SILMA-Kashif-2B-Instruct-v1.0-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
SILMA-Kashif-2B-Instruct-v1.0-i1-Q6_K-GGUF
SILMA-Kashif-2B-Instruct-v1.0-i1-IQ4_XS-GGUF
14B-Qwen2.5-Kunou-v1-Q6_K-GGUF
q-2.5-deepseek-r1-veltha-v0.3-Q3_K_M-GGUF
Lascivious-LLaMa-70B-Q4_K_S-GGUF
Virtuoso-Lite-Q2_K-GGUF
Virtuoso-Lite-Q8_0-GGUF
Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q2_K-GGUF
DS-R1-Distill-Q2.5-14B-Harmony_V0.1-Q8_0-GGUF
roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q80-GGUF Repo: `roleplaiapp/DS-R1-Distill-Q2.5-14B-HarmonyV0.1-Q80-GGUF` Original Model: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1` Quantized File: `DS-R1-Distill-Q2.5-14B-HarmonyV0.1.Q80.gguf` Quantization: `GGUF` Quantization Method: `Q80` Overview This is a GGUF Q80 quantized version of DS-R1-Distill-Q2.5-14B-HarmonyV0.1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
MN-12B-Mimicore-Orochi-Q3_K_S-GGUF
Llama3.2-doker-Q5_K_M-GGUF
Jaja-small-v1-Q2_K-GGUF
Repo: `roleplaiapp/Jaja-small-v1-Q2K-GGUF` Original Model: `Jaja-small-v1` Quantized File: `Jaja-small-v1.Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of Jaja-small-v1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Slush-Sunfall-Rocinante-GGLD-12B-IQ4_XS-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q2_K-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q2K-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q2K-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
cyberagent-DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q2_K-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q4_K_M-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q5_K_S-GGUF
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S-GGUF
roleplaiapp/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5KS-GGUF Repo: `roleplaiapp/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5KS-GGUF` Original Model: `FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview` Quantized File: `FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5KS.gguf` Quantization: `GGUF` Quantization Method: `Q5KS` Overview This is a GGUF Q5KS quantized version of FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS-GGUF
Qwen2.5-32B-DeepSeek-R1-Instruct-i1-IQ3_XS-GGUF
roleplaiapp/Qwen2.5-32B-DeepSeek-R1-Instruct-i1-IQ3XS-GGUF Repo: `roleplaiapp/Qwen2.5-32B-DeepSeek-R1-Instruct-i1-IQ3XS-GGUF` Original Model: `Qwen2.5-32B-DeepSeek-R1-Instruct-i1` Quantized File: `Qwen2.5-32B-DeepSeek-R1-Instruct.i1-IQ3XS.gguf` Quantization: `GGUF` Quantization Method: `IQ3XS` Overview This is a GGUF IQ3XS quantized version of Qwen2.5-32B-DeepSeek-R1-Instruct-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
YuE-s1-7B-anneal-en-cot-Q8_0-GGUF
RuadaptQwen2.5-32B-Pro-Beta-Q8_0-GGUF
Cydonia-24B-v2a-Q6_K-GGUF
Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-Q8_0-GGUF
Confucius-o1-14B-Q8_0-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q3_K_S-GGUF
SmallThinker-3B-Preview-Q2_K-GGUF
SmallThinker-3B-Preview-Q3_K_M-GGUF
QwQ-32B-Preview-Q5_K_S-GGUF
internlm3-8b-instruct-Q3_K_L-GGUF
ReaderLM-v2-Q5_K_S-GGUF
Dria-Agent-a-7B-Q8_0-GGUF
Llama-3.1-Nemotron-70B-Instruct-HF-Q3_K_L-GGUF
AceInstruct-72B-Q8_0-GGUF
AceInstruct-7B-Q6_K-GGUF
Repo: `roleplaiapp/AceInstruct-7B-Q6K-GGUF` Original Model: `AceInstruct-7B` Organization: `nvidia` Quantized File: `aceinstruct-7b-q6k.gguf` Quantization: `GGUF` Quantization Method: `Q6K` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q6K quantized version of AceInstruct-7B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Llama-3.1-Nemotron-70B-Instruct-HF-Q6_K-GGUF
DeepSeek-R1-Distill-Llama-8B-Q4_K_S-GGUF
DeepSeek-R1-Distill-Llama-8B-Q5_K_M-GGUF
DeepSeek-R1-Distill-Qwen-7B-Q8_0-GGUF
ALIA-40b-Q6_K-GGUF
MN-12B-Mag-Mell-R1-IQ4_XS-GGUF
Repo: `roleplaiapp/MN-12B-Mag-Mell-R1-IQ4XS-GGUF` Original Model: `MN-12B-Mag-Mell-R1` Quantized File: `MN-12B-Mag-Mell-R1.IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of MN-12B-Mag-Mell-R1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Virtuoso-Lite-Q4_K_S-GGUF
Pathfinder-RP-12B-RU-Q8_0-GGUF
Pathfinder-RP-12B-RU-IQ4_XS-GGUF
Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q5_K_M-GGUF
roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q5KM-GGUF Repo: `roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q5KM-GGUF` Original Model: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored` Quantized File: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored.Q5KM.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Overview This is a GGUF Q5KM quantized version of Qwen2.5-14B-DeepSeek-R1-1M-Uncensored Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
MN-12B-Mimicore-Orochi-Q3_K_L-GGUF
GRAG-R1-14B-SFT-DE-EXP-Q8_0-GGUF
GRAG-R1-14B-SFT-DE-EXP-Q3_K_S-GGUF
DialoGPT-medium-anand-Q6_K-GGUF
DialoGPT-medium-anand-Q3_K_L-GGUF
DialoGPT-medium-anand-f16-GGUF
Repo: `roleplaiapp/DialoGPT-medium-anand-f16-GGUF` Original Model: `DialoGPT-medium-anand` Quantized File: `DialoGPT-medium-anand.f16.gguf` Quantization: `GGUF` Quantization Method: `f16` Overview This is a GGUF f16 quantized version of DialoGPT-medium-anand Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Jaja-small-v1-Q3_K_S-GGUF
Repo: `roleplaiapp/Jaja-small-v1-Q3KS-GGUF` Original Model: `Jaja-small-v1` Quantized File: `Jaja-small-v1.Q3KS.gguf` Quantization: `GGUF` Quantization Method: `Q3KS` Overview This is a GGUF Q3KS quantized version of Jaja-small-v1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-Small-24B-Instruct-2501-Q3_K_L-GGUF
Reasoning-Llama-3.1-CoT-RE1-Q6_K-GGUF
Repo: `roleplaiapp/Reasoning-Llama-3.1-CoT-RE1-Q6K-GGUF` Original Model: `Reasoning-Llama-3.1-CoT-RE1` Quantized File: `Reasoning-Llama-3.1-CoT-RE1.Q6K.gguf` Quantization: `GGUF` Quantization Method: `Q6K` Overview This is a GGUF Q6K quantized version of Reasoning-Llama-3.1-CoT-RE1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q5_K_M-GGUF
roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q5KM-GGUF Repo: `roleplaiapp/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B-Q5KM-GGUF` Original Model: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B` Quantized File: `DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B.Q5KM.gguf` Quantization: `GGUF` Quantization Method: `Q5KM` Overview This is a GGUF Q5KM quantized version of DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Llama-3-monika-ddlc-11.5b-v1-i1-IQ3_S-GGUF
L3.3-Nevoria-R1-70b-Q4_K_S-GGUF
medicine-LLM-Q5_K_S-GGUF
DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q4_K_M-GGUF
DeepSeek-R1-Distill-Qwen-32B-Japanese-gguf-Q8_0-GGUF
Confucius-o1-14B-Q2_K-GGUF
plato-9b-Q4_K_S-GGUF
Repo: `roleplaiapp/plato-9b-Q4KS-GGUF` Original Model: `plato-9b` Quantized File: `plato-9b.Q4KS.gguf` Quantization: `GGUF` Quantization Method: `Q4KS` Overview This is a GGUF Q4KS quantized version of plato-9b Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Janus-Pro-7B-LM-Q4_K_S-GGUF
SmallThinker-3B-Preview-Q3_K_L-GGUF
QwQ-32B-Preview-Q3_K_L-GGUF
Repo: `roleplaiapp/QwQ-32B-Preview-Q3KL-GGUF` Original Model: `QwQ-32B-Preview` Organization: `Qwen` Quantized File: `qwq-32b-preview-q3kl.gguf` Quantization: `GGUF` Quantization Method: `Q3KL` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q3KL quantized version of QwQ-32B-Preview. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
internlm3-8b-instruct-Q5_K_S-GGUF
internlm3-8b-instruct-Q5_K_M-GGUF
internlm3-8b-instruct-Q8_0-GGUF
Dria-Agent-a-3B-Q2_K-GGUF
Dria-Agent-a-3B-Q5_K_M-GGUF
Wayfarer-12B-Q3_K_L-GGUF
Repo: `roleplaiapp/Wayfarer-12B-Q3KL-GGUF` Original Model: `Wayfarer-12B` Organization: `LatitudeGames` Quantized File: `wayfarer-12b-q3kl.gguf` Quantization: `GGUF` Quantization Method: `Q3KL` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q3KL quantized version of Wayfarer-12B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Wayfarer-12B-Q5_K_S-GGUF
Wayfarer-12B-Q6_K-GGUF
Codestral-22B-v0.1-Q3_K_L-GGUF
AceInstruct-1.5B-Q3_K_L-GGUF
Omni-Reasoner-2B-Q3_K_L-GGUF
Repo: `roleplaiapp/Omni-Reasoner-2B-Q3KL-GGUF` Original Model: `Omni-Reasoner-o1` Organization: `prithivMLmods` Quantized File: `omni-reasoner-2b-q3kl.gguf` Quantization: `GGUF` Quantization Method: `Q3KL` Use Imatrix: `False` Split Model: `False` Overview This is an GGUF Q3KL quantized version of Omni-Reasoner-o1. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Omni-Reasoner-2B-Q5_K_S-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q5_0-GGUF
DeepSeek-R1-Distill-Qwen-1.5B-Q5_K_M-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q3_K_L-GGUF
Qwen2.5-7B-Instruct-Uncensored-Q3_K_S-GGUF
QwQ-32B-Preview-abliterated-Q2_K-GGUF
Midnight-Miqu-70B-v1.5-i1-IQ3_M-GGUF
DS-Distilled-Hermes-Llama-3.1_TIES-i1-IQ3_M-GGUF
DS-Distilled-Hermes-Llama-3.1_TIES-i1-IQ3_S-GGUF
Phi-4-ReasoningRP-Q8_0-GGUF
Virtuoso-Lite-Q6_K-GGUF
Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q4_K_S-GGUF
roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q4KS-GGUF Repo: `roleplaiapp/Qwen2.5-14B-DeepSeek-R1-1M-Uncensored-Q4KS-GGUF` Original Model: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored` Quantized File: `Qwen2.5-14B-DeepSeek-R1-1M-Uncensored.Q4KS.gguf` Quantization: `GGUF` Quantization Method: `Q4KS` Overview This is a GGUF Q4KS quantized version of Qwen2.5-14B-DeepSeek-R1-1M-Uncensored Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Minerva-14b-V0.1-i1-Q4_K_M-GGUF
Llama3.2-doker-Q8_0-GGUF
Llama3.2-doker-f16-GGUF
DialoGPT-medium-anand-IQ4_XS-GGUF
Repo: `roleplaiapp/DialoGPT-medium-anand-IQ4XS-GGUF` Original Model: `DialoGPT-medium-anand` Quantized File: `DialoGPT-medium-anand.IQ4XS.gguf` Quantization: `GGUF` Quantization Method: `IQ4XS` Overview This is a GGUF IQ4XS quantized version of DialoGPT-medium-anand Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Mistral-Small-24B-Instruct-2501-f16-GGUF
roleplaiapp/Mistral-Small-24B-Instruct-2501-f16-GGUF Repo: `roleplaiapp/Mistral-Small-24B-Instruct-2501-f16-GGUF` Original Model: `Mistral-Small-24B-Instruct-2501` Quantized File: `Mistral-Small-24B-Instruct-2501-f16.gguf` Quantization: `GGUF` Quantization Method: `f16` Overview This is a GGUF f16 quantized version of Mistral-Small-24B-Instruct-2501 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Reasoning-Llama-3.1-CoT-RE1-Q2_K-GGUF
Repo: `roleplaiapp/Reasoning-Llama-3.1-CoT-RE1-Q2K-GGUF` Original Model: `Reasoning-Llama-3.1-CoT-RE1` Quantized File: `Reasoning-Llama-3.1-CoT-RE1.Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of Reasoning-Llama-3.1-CoT-RE1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Qwen2.5-Coder-14B-Instruct-Uncensored-Q8_0-GGUF
TunnedLlama-3.1-8B_v2-Q3_K_L-GGUF
Repo: `roleplaiapp/TunnedLlama-3.1-8Bv2-Q3KL-GGUF` Original Model: `TunnedLlama-3.1-8Bv2` Quantized File: `TunnedLlama-3.1-8Bv2.Q3KL.gguf` Quantization: `GGUF` Quantization Method: `Q3KL` Overview This is a GGUF Q3KL quantized version of TunnedLlama-3.1-8Bv2 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Qwen2.5-1.5B-Open-R1-Distill-Q3_K_L-GGUF
Qwen2.5-7B-olm-v1.4-i1-IQ3_S-GGUF
deepseek-r1-qwen-2.5-32B-ablated-Q8_0-GGUF
roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q80-GGUF Repo: `roleplaiapp/deepseek-r1-qwen-2.5-32B-ablated-Q80-GGUF` Original Model: `deepseek-r1-qwen-2.5-32B-ablated` Quantized File: `deepseek-r1-qwen-2.5-32B-ablated-Q80.gguf` Quantization: `GGUF` Quantization Method: `Q80` Overview This is a GGUF Q80 quantized version of deepseek-r1-qwen-2.5-32B-ablated Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
cyberagent-DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q6_K-GGUF
cyberagent-DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q8_0-GGUF
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L-GGUF
Qwen2.5-7B-Instruct-1M-IQ3_XS-GGUF
DeepSky-T100-Q6_K-GGUF
Llama-3-monika-ddlc-11.5b-v1-i1-Q4_K_M-GGUF
Llama-3-monika-ddlc-11.5b-v1-i1-IQ4_XS-GGUF
R1-Qwen2.5-32B-Instruct-1k-Q3_K_S-GGUF
YuE-s1-7B-anneal-en-cot-Q3_K_S-GGUF
L3.3-Nevoria-R1-70b-Q8_0-GGUF
Pantheon-RP-Pure-1.6.2-22b-Small-Q8_0-GGUF
DeepSeek-R1-Distill-Qwen-14B-Japanese-gguf-Q8_0-GGUF
saiga_nemo_12b_gguf-Q3_K_S-GGUF
saiga_nemo_12b_gguf-Q4_K_M-GGUF
Confucius-o1-14B-IQ3_M-GGUF
DeepSeek-R1-Distill-sthenno-14b-0121-Q8_0-GGUF
SJT-Moe2x7.5B-IQ4_XS-GGUF
QwQ-32B-Preview-Q4_K_M-GGUF
Dria-Agent-a-7B-Q4_K_M-GGUF
phi-4-4.0bpw-exl2
SmallThinker-3B-Preview-Q5_K_S-GGUF
QwQ-32B-Preview-Q4_K_S-GGUF
Dria-Agent-a-7B-Q4_0-GGUF
Dria-Agent-a-7B-Q5_0-GGUF
AceInstruct-72B-Q5_0-GGUF
Repo: `roleplaiapp/AceInstruct-72B-Q50-GGUF` Original Model: `AceInstruct-72B` Organization: `nvidia` Quantized File: `aceinstruct-72b-q50.gguf` Quantization: `GGUF` Quantization Method: `Q50` Use Imatrix: `False` Split Model: `True` Overview This is an GGUF Q50 quantized version of AceInstruct-72B. Quantization By I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.
Llama-3.1-Nemotron-70B-Instruct-HF-Q4_K_S-GGUF
DeepSeek-R1-Distill-Qwen-32B-Q3_K_M-GGUF
DS-Distilled-Hermes-Llama-3.1_TIES-i1-Q5_K_M-GGUF
DS-Distilled-Hermes-Llama-3.1_TIES-i1-IQ4_XS-GGUF
Reasoning-Llama-3.1-CoT-RE1-i1-Q2_K-GGUF
roleplaiapp/Reasoning-Llama-3.1-CoT-RE1-i1-Q2K-GGUF Repo: `roleplaiapp/Reasoning-Llama-3.1-CoT-RE1-i1-Q2K-GGUF` Original Model: `Reasoning-Llama-3.1-CoT-RE1-i1` Quantized File: `Reasoning-Llama-3.1-CoT-RE1.i1-Q2K.gguf` Quantization: `GGUF` Quantization Method: `Q2K` Overview This is a GGUF Q2K quantized version of Reasoning-Llama-3.1-CoT-RE1-i1 Quantization By I often have idle GPUs while building/testing for the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.