mradermacher

static quants of https://huggingface.co/inflatebot/MN-12B-Mag-Mell-R1 weighted/imatrix quants are available at https://huggingface.co/mradermacher/MN-12B-Mag-Mell-R1-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q2K | 4.9 | | | GGUF | IQ3XS | 5.4 | | | GGUF | Q3KS | 5.6 | | | GGUF | IQ3S | 5.7 | beats Q3K | | GGUF | IQ3M | 5.8 | | | GGUF | Q3KM | 6.2 | lower quality | | GGUF | Q3KL | 6.7 | | | GGUF | IQ4XS | 6.9 | | | GGUF | Q4KS | 7.2 | fast, recommended | | GGUF | Q4KM | 7.6 | fast, recommended | | GGUF | Q5KS | 8.6 | | | GGUF | Q5KM | 8.8 | | | GGUF | Q6K | 10.2 | very good quality | | GGUF | Q80 | 13.1 | fast, best quality | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

weighted/imatrix quants of https://huggingface.co/p-e-w/Llama-3.1-8B-Instruct-heretic For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/Llama-3.1-8B-Instruct-heretic-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | imatrix | 0.1 | imatrix file (for creating your own qwuants) | | GGUF | i1-IQ1S | 2.1 | for the desperate | | GGUF | i1-IQ1M | 2.3 | mostly desperate | | GGUF | i1-IQ2XXS | 2.5 | | | GGUF | i1-IQ2XS | 2.7 | | | GGUF | i1-IQ2S | 2.9 | | | GGUF | i1-IQ2M | 3.0 | | | GGUF | i1-Q2KS | 3.1 | very low quality | | GGUF | i1-Q2K | 3.3 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 3.4 | lower quality | | GGUF | i1-IQ3XS | 3.6 | | | GGUF | i1-Q3KS | 3.8 | IQ3XS probably better | | GGUF | i1-IQ3S | 3.8 | beats Q3K | | GGUF | i1-IQ3M | 3.9 | | | GGUF | i1-Q3KM | 4.1 | IQ3S probably better | | GGUF | i1-Q3KL | 4.4 | IQ3M probably better | | GGUF | i1-IQ4XS | 4.5 | | | GGUF | i1-Q40 | 4.8 | fast, low quality | | GGUF | i1-IQ4NL | 4.8 | prefer IQ4XS | | GGUF | i1-Q4KS | 4.8 | optimal size/speed/quality | | GGUF | i1-Q4KM | 5.0 | fast, recommended | | GGUF | i1-Q41 | 5.2 | | | GGUF | i1-Q5KS | 5.7 | | | GGUF | i1-Q5KM | 5.8 | | | GGUF | i1-Q6K | 6.7 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

weighted/imatrix quants of https://huggingface.co/Statuo/Deepseeker-Kunou-Qwen2.5-14b static quants are available at https://huggingface.co/mradermacher/Deepseeker-Kunou-Qwen2.5-14b-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | i1-IQ1S | 3.7 | for the desperate | | GGUF | i1-IQ1M | 4.0 | mostly desperate | | GGUF | i1-IQ2XXS | 4.4 | | | GGUF | i1-IQ2XS | 4.8 | | | GGUF | i1-IQ2S | 5.1 | | | GGUF | i1-IQ2M | 5.5 | | | GGUF | i1-Q2KS | 5.5 | very low quality | | GGUF | i1-Q2K | 5.9 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 6.0 | lower quality | | GGUF | i1-IQ3XS | 6.5 | | | GGUF | i1-Q3KS | 6.8 | IQ3XS probably better | | GGUF | i1-IQ3S | 6.8 | beats Q3K | | GGUF | i1-IQ3M | 7.0 | | | GGUF | i1-Q3KM | 7.4 | IQ3S probably better | | GGUF | i1-Q3KL | 8.0 | IQ3M probably better | | GGUF | i1-IQ4XS | 8.2 | | | GGUF | i1-Q40 | 8.6 | fast, low quality | | GGUF | i1-IQ4NL | 8.6 | prefer IQ4XS | | GGUF | i1-Q4KS | 8.7 | optimal size/speed/quality | | GGUF | i1-Q4KM | 9.1 | fast, recommended | | GGUF | i1-Q41 | 9.5 | | | GGUF | i1-Q5KS | 10.4 | | | GGUF | i1-Q5KM | 10.6 | | | GGUF | i1-Q6K | 12.2 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

static quants of https://huggingface.co/nanonets/Nanonets-OCR2-3B For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Nanonets-OCR2-3B-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | mmproj-Q80 | 0.9 | multi-modal supplement | | GGUF | Q2K | 1.4 | | | GGUF | mmproj-f16 | 1.4 | multi-modal supplement | | GGUF | Q3KS | 1.6 | | | GGUF | Q3KM | 1.7 | lower quality | | GGUF | Q3KL | 1.8 | | | GGUF | IQ4XS | 1.9 | | | GGUF | Q4KS | 1.9 | fast, recommended | | GGUF | Q4KM | 2.0 | fast, recommended | | GGUF | Q5KS | 2.3 | | | GGUF | Q5KM | 2.3 | | | GGUF | Q6K | 2.6 | very good quality | | GGUF | Q80 | 3.4 | fast, best quality | | GGUF | f16 | 6.3 | 16 bpw, overkill | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

static quants of https://huggingface.co/huizimao/gpt-oss-20b-uncensored-bf16 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/gpt-oss-20b-uncensored-bf16-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q3KS | 12.2 | | | GGUF | Q2K | 12.2 | | | GGUF | IQ4XS | 12.3 | | | GGUF | Q3KM | 13.0 | lower quality | | GGUF | Q3KL | 13.4 | | | GGUF | Q4KS | 14.8 | fast, recommended | | GGUF | Q4KM | 15.9 | fast, recommended | | GGUF | Q5KS | 16.0 | | | GGUF | Q5KM | 17.0 | | | GGUF | Q6K | 22.3 | very good quality | | GGUF | Q80 | 22.4 | fast, best quality | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

static quants of https://huggingface.co/huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Qwen2.5-VL-7B-Instruct-abliterated-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | mmproj-Q80 | 1.0 | multi-modal supplement | | GGUF | mmproj-f16 | 1.5 | multi-modal supplement | | GGUF | Q2K | 3.1 | | | GGUF | Q3KS | 3.6 | | | GGUF | Q3KM | 3.9 | lower quality | | GGUF | Q3KL | 4.2 | | | GGUF | IQ4XS | 4.4 | | | GGUF | Q4KS | 4.6 | fast, recommended | | GGUF | Q4KM | 4.8 | fast, recommended | | GGUF | Q5KS | 5.4 | | | GGUF | Q5KM | 5.5 | | | GGUF | Q6K | 6.4 | very good quality | | GGUF | Q80 | 8.2 | fast, best quality | | GGUF | f16 | 15.3 | 16 bpw, overkill | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

static quants of https://huggingface.co/nicoboss/DeepSeek-R1-Distill-Qwen-14B-Uncensored weighted/imatrix quants are available at https://huggingface.co/mradermacher/DeepSeek-R1-Distill-Qwen-14B-Uncensored-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q2K | 5.9 | | | GGUF | Q3KS | 6.8 | | | GGUF | Q3KM | 7.4 | lower quality | | GGUF | Q3KL | 8.0 | | | GGUF | IQ4XS | 8.3 | | | GGUF | Q4KS | 8.7 | fast, recommended | | GGUF | Q4KM | 9.1 | fast, recommended | | GGUF | Q5KS | 10.4 | | | GGUF | Q5KM | 10.6 | | | GGUF | Q6K | 12.2 | very good quality | | GGUF | Q80 | 15.8 | fast, best quality | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

weighted/imatrix quants of https://huggingface.co/FlareRebellion/WeirdCompound-v1.7-24b For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/WeirdCompound-v1.7-24b-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | imatrix | 0.1 | imatrix file (for creating your own qwuants) | | GGUF | i1-IQ1S | 5.4 | for the desperate | | GGUF | i1-IQ1M | 5.9 | mostly desperate | | GGUF | i1-IQ2XXS | 6.6 | | | GGUF | i1-IQ2XS | 7.3 | | | GGUF | i1-IQ2S | 7.6 | | | GGUF | i1-IQ2M | 8.2 | | | GGUF | i1-Q2KS | 8.4 | very low quality | | GGUF | i1-Q2K | 9.0 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 9.4 | lower quality | | GGUF | i1-IQ3XS | 10.0 | | | GGUF | i1-Q3KS | 10.5 | IQ3XS probably better | | GGUF | i1-IQ3S | 10.5 | beats Q3K | | GGUF | i1-IQ3M | 10.8 | | | GGUF | i1-Q3KM | 11.6 | IQ3S probably better | | GGUF | i1-Q3KL | 12.5 | IQ3M probably better | | GGUF | i1-IQ4XS | 12.9 | | | GGUF | i1-Q40 | 13.6 | fast, low quality | | GGUF | i1-Q4KS | 13.6 | optimal size/speed/quality | | GGUF | i1-Q4KM | 14.4 | fast, recommended | | GGUF | i1-Q41 | 15.0 | | | GGUF | i1-Q5KS | 16.4 | | | GGUF | i1-Q5KM | 16.9 | | | GGUF | i1-Q6K | 19.4 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

static quants of https://huggingface.co/prithivMLmods/Qwen2.5-VL-7B-Abliterated-Caption-it For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Qwen2.5-VL-7B-Abliterated-Caption-it-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | mmproj-Q80 | 1.0 | multi-modal supplement | | GGUF | mmproj-f16 | 1.5 | multi-modal supplement | | GGUF | Q2K | 3.1 | | | GGUF | Q3KS | 3.6 | | | GGUF | Q3KM | 3.9 | lower quality | | GGUF | Q3KL | 4.2 | | | GGUF | IQ4XS | 4.4 | | | GGUF | Q4KS | 4.6 | fast, recommended | | GGUF | Q4KM | 4.8 | fast, recommended | | GGUF | Q5KS | 5.4 | | | GGUF | Q5KM | 5.5 | | | GGUF | Q6K | 6.4 | very good quality | | GGUF | Q80 | 8.2 | fast, best quality | | GGUF | f16 | 15.3 | 16 bpw, overkill | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

weighted/imatrix quants of https://huggingface.co/prithivMLmods/Qwen3-VL-8B-Abliterated-Caption-it For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/Qwen3-VL-8B-Abliterated-Caption-it-GGUF This is a vision model - mmproj files (if any) will be in the static repository. Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | imatrix | 0.1 | imatrix file (for creating your own qwuants) | | GGUF | i1-IQ1S | 2.2 | for the desperate | | GGUF | i1-IQ1M | 2.4 | mostly desperate | | GGUF | i1-IQ2XXS | 2.6 | | | GGUF | i1-IQ2XS | 2.8 | | | GGUF | i1-IQ2S | 3.0 | | | GGUF | i1-IQ2M | 3.2 | | | GGUF | i1-Q2KS | 3.2 | very low quality | | GGUF | i1-Q2K | 3.4 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 3.5 | lower quality | | GGUF | i1-IQ3XS | 3.7 | | | GGUF | i1-Q3KS | 3.9 | IQ3XS probably better | | GGUF | i1-IQ3S | 3.9 | beats Q3K | | GGUF | i1-IQ3M | 4.0 | | | GGUF | i1-Q3KM | 4.2 | IQ3S probably better | | GGUF | i1-Q3KL | 4.5 | IQ3M probably better | | GGUF | i1-IQ4XS | 4.7 | | | GGUF | i1-Q40 | 4.9 | fast, low quality | | GGUF | i1-IQ4NL | 4.9 | prefer IQ4XS | | GGUF | i1-Q4KS | 4.9 | optimal size/speed/quality | | GGUF | i1-Q4KM | 5.1 | fast, recommended | | GGUF | i1-Q41 | 5.3 | | | GGUF | i1-Q5KS | 5.8 | | | GGUF | i1-Q5KM | 6.0 | | | GGUF | i1-Q6K | 6.8 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

static quants of https://huggingface.co/DavidAU/Qwen3-42B-A3B-2507-Thinking-TOTAL-RECALL-v2-Medium-MASTER-CODER For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Qwen3-42B-A3B-2507-Thinking-TOTAL-RECALL-v2-Medium-MASTER-CODER-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q2K | 15.7 | | | GGUF | Q3KS | 18.5 | | | GGUF | Q3KM | 20.5 | lower quality | | GGUF | Q3KL | 22.1 | | | GGUF | IQ4XS | 23.0 | | | GGUF | Q4KS | 24.3 | fast, recommended | | GGUF | Q4KM | 25.8 | fast, recommended | | GGUF | Q5KS | 29.3 | | | GGUF | Q5KM | 30.2 | | | GGUF | Q6K | 34.9 | very good quality | | GGUF | Q80 | 45.2 | fast, best quality | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

weighted/imatrix quants of https://huggingface.co/DavidAU/Qwen3-42B-A3B-2507-Thinking-Abliterated-uncensored-TOTAL-RECALL-v2-Medium-MASTER-CODER For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/Qwen3-42B-A3B-2507-Thinking-Abliterated-uncensored-TOTAL-RECALL-v2-Medium-MASTER-CODER-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | imatrix | 0.3 | imatrix file (for creating your own qwuants) | | GGUF | i1-IQ1S | 8.9 | for the desperate | | GGUF | i1-IQ1M | 9.8 | mostly desperate | | GGUF | i1-IQ2XXS | 11.4 | | | GGUF | i1-IQ2XS | 12.6 | | | GGUF | i1-IQ2S | 12.9 | | | GGUF | i1-IQ2M | 14.1 | | | GGUF | i1-Q2KS | 14.6 | very low quality | | GGUF | i1-Q2K | 15.7 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 16.5 | lower quality | | GGUF | i1-IQ3XS | 17.5 | | | GGUF | i1-Q3KS | 18.5 | IQ3XS probably better | | GGUF | i1-IQ3S | 18.5 | beats Q3K | | GGUF | i1-IQ3M | 18.8 | | | GGUF | i1-Q3KM | 20.5 | IQ3S probably better | | GGUF | i1-Q3KL | 22.1 | IQ3M probably better | | GGUF | i1-IQ4XS | 22.8 | | | GGUF | i1-Q40 | 24.2 | fast, low quality | | GGUF | i1-Q4KS | 24.3 | optimal size/speed/quality | | GGUF | i1-Q4KM | 25.8 | fast, recommended | | GGUF | i1-Q41 | 26.7 | | | GGUF | i1-Q5KS | 29.3 | | | GGUF | i1-Q5KM | 30.2 | | | GGUF | i1-Q6K | 34.9 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

static quants of https://huggingface.co/huihui-ai/Huihui-Qwen3-30B-A3B-Instruct-2507-abliterated For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Huihui-Qwen3-30B-A3B-Instruct-2507-abliterated-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q2K | 11.4 | | | GGUF | Q3KS | 13.4 | | | GGUF | Q3KM | 14.8 | lower quality | | GGUF | Q3KL | 16.0 | | | GGUF | IQ4XS | 16.7 | | | GGUF | Q4KS | 17.6 | fast, recommended | | GGUF | Q4KM | 18.7 | fast, recommended | | GGUF | Q5KS | 21.2 | | | GGUF | Q5KM | 21.8 | | | GGUF | Q6K | 25.2 | very good quality | | GGUF | Q80 | 32.6 | fast, best quality | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

static quants of https://huggingface.co/chuanli11/Llama-3.2-3B-Instruct-uncensored weighted/imatrix quants are available at https://huggingface.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q2K | 1.6 | | | GGUF | IQ3XS | 1.7 | | | GGUF | IQ3S | 1.8 | beats Q3K | | GGUF | Q3KS | 1.8 | | | GGUF | IQ3M | 1.9 | | | GGUF | Q3KM | 2.0 | lower quality | | GGUF | Q3KL | 2.1 | | | GGUF | IQ4XS | 2.2 | | | GGUF | Q4KS | 2.2 | fast, recommended | | GGUF | Q4KM | 2.3 | fast, recommended | | GGUF | Q5KS | 2.6 | | | GGUF | Q5KM | 2.7 | | | GGUF | Q6K | 3.1 | very good quality | | GGUF | Q80 | 3.9 | fast, best quality | | GGUF | f16 | 7.3 | 16 bpw, overkill | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

static quants of https://huggingface.co/BruhzWater/Sapphira-L3.3-70b-0.2 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Sapphira-L3.3-70b-0.2-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q2K | 26.5 | | | GGUF | Q3KS | 31.0 | | | GGUF | Q3KM | 34.4 | lower quality | | GGUF | Q3KL | 37.2 | | | GGUF | IQ4XS | 38.4 | | | GGUF | Q4KS | 40.4 | fast, recommended | | GGUF | Q4KM | 42.6 | fast, recommended | | GGUF | Q5KS | 48.8 | | | GGUF | Q5KM | 50.0 | | | PART 1 PART 2 | Q6K | 58.0 | very good quality | | PART 1 PART 2 | Q80 | 75.1 | fast, best quality | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

weighted/imatrix quants of https://huggingface.co/bountyhunterxx/glm4.1v-9b-base-sft For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/glm4.1v-9b-base-sft-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | imatrix | 0.1 | imatrix file (for creating your own qwuants) | | GGUF | i1-IQ1S | 3.2 | for the desperate | | GGUF | i1-IQ1M | 3.3 | mostly desperate | | GGUF | i1-IQ2XXS | 3.5 | | | GGUF | i1-IQ2XS | 3.7 | | | GGUF | i1-IQ2S | 3.9 | | | GGUF | i1-IQ2M | 4.0 | | | GGUF | i1-Q2KS | 4.1 | very low quality | | GGUF | i1-Q2K | 4.1 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 4.3 | lower quality | | GGUF | i1-IQ3XS | 4.5 | | | GGUF | i1-Q3KS | 4.7 | IQ3XS probably better | | GGUF | i1-IQ3S | 4.7 | beats Q3K | | GGUF | i1-IQ3M | 4.8 | | | GGUF | i1-Q3KM | 5.1 | IQ3S probably better | | GGUF | i1-Q3KL | 5.3 | IQ3M probably better | | GGUF | i1-IQ4XS | 5.4 | | | GGUF | i1-IQ4NL | 5.6 | prefer IQ4XS | | GGUF | i1-Q40 | 5.6 | fast, low quality | | GGUF | i1-Q4KS | 5.9 | optimal size/speed/quality | | GGUF | i1-Q41 | 6.1 | | | GGUF | i1-Q4KM | 6.3 | fast, recommended | | GGUF | i1-Q5KS | 6.8 | | | GGUF | i1-Q5KM | 7.2 | | | GGUF | i1-Q6K | 8.4 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

static quants of https://huggingface.co/Liontix/Qwen3-4B-Claude-Sonnet-4-Reasoning-Distill-Safetensor For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | Q2K | 1.8 | | | GGUF | Q3KS | 2.0 | | | GGUF | Q3KM | 2.2 | lower quality | | GGUF | Q3KL | 2.3 | | | GGUF | IQ4XS | 2.4 | | | GGUF | Q4KS | 2.5 | fast, recommended | | GGUF | Q4KM | 2.6 | fast, recommended | | GGUF | Q5KS | 2.9 | | | GGUF | Q5KM | 3.0 | | | GGUF | Q6K | 3.4 | very good quality | | GGUF | Q80 | 4.4 | fast, best quality | | GGUF | f16 | 8.2 | 16 bpw, overkill | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

weighted/imatrix quants of https://huggingface.co/prithivMLmods/Gliese-OCR-7B-Post2.0-final For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/Gliese-OCR-7B-Post2.0-final-GGUF This is a vision model - mmproj files (if any) will be in the static repository. Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | imatrix | 0.1 | imatrix file (for creating your own qwuants) | | GGUF | i1-IQ1S | 2.0 | for the desperate | | GGUF | i1-IQ1M | 2.1 | mostly desperate | | GGUF | i1-IQ2XXS | 2.4 | | | GGUF | i1-IQ2XS | 2.6 | | | GGUF | i1-IQ2S | 2.7 | | | GGUF | i1-IQ2M | 2.9 | | | GGUF | i1-Q2KS | 2.9 | very low quality | | GGUF | i1-Q2K | 3.1 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 3.2 | lower quality | | GGUF | i1-IQ3XS | 3.4 | | | GGUF | i1-Q3KS | 3.6 | IQ3XS probably better | | GGUF | i1-IQ3S | 3.6 | beats Q3K | | GGUF | i1-IQ3M | 3.7 | | | GGUF | i1-Q3KM | 3.9 | IQ3S probably better | | GGUF | i1-Q3KL | 4.2 | IQ3M probably better | | GGUF | i1-IQ4XS | 4.3 | | | GGUF | i1-IQ4NL | 4.5 | prefer IQ4XS | | GGUF | i1-Q40 | 4.5 | fast, low quality | | GGUF | i1-Q4KS | 4.6 | optimal size/speed/quality | | GGUF | i1-Q4KM | 4.8 | fast, recommended | | GGUF | i1-Q41 | 5.0 | | | GGUF | i1-Q5KS | 5.4 | | | GGUF | i1-Q5KM | 5.5 | | | GGUF | i1-Q6K | 6.4 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

static quants of https://huggingface.co/thesby/Qwen2.5-VL-7B-NSFW-Caption-V4 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Qwen2.5-VL-7B-NSFW-Caption-V4-i1-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | mmproj-Q80 | 0.8 | multi-modal supplement | | GGUF | mmproj-f16 | 1.5 | multi-modal supplement | | GGUF | Q2K | 3.2 | | | GGUF | Q3KS | 3.6 | | | GGUF | Q3KM | 4.0 | lower quality | | GGUF | Q3KL | 4.2 | | | GGUF | IQ4XS | 4.4 | | | GGUF | Q4KS | 4.6 | fast, recommended | | GGUF | Q4KM | 4.8 | fast, recommended | | GGUF | Q5KS | 5.4 | | | GGUF | Q5KM | 5.5 | | | GGUF | Q6K | 6.4 | very good quality | | GGUF | Q80 | 8.2 | fast, best quality | | GGUF | f16 | 15.4 | 16 bpw, overkill | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

weighted/imatrix quants of https://huggingface.co/vanta-research/wraith-8b For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/wraith-8b-GGUF Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | imatrix | 0.1 | imatrix file (for creating your own qwuants) | | GGUF | i1-IQ1S | 2.1 | for the desperate | | GGUF | i1-IQ1M | 2.3 | mostly desperate | | GGUF | i1-IQ2XXS | 2.5 | | | GGUF | i1-IQ2XS | 2.7 | | | GGUF | i1-IQ2S | 2.9 | | | GGUF | i1-IQ2M | 3.0 | | | GGUF | i1-Q2KS | 3.1 | very low quality | | GGUF | i1-Q2K | 3.3 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 3.4 | lower quality | | GGUF | i1-IQ3XS | 3.6 | | | GGUF | i1-Q3KS | 3.8 | IQ3XS probably better | | GGUF | i1-IQ3S | 3.8 | beats Q3K | | GGUF | i1-IQ3M | 3.9 | | | GGUF | i1-Q3KM | 4.1 | IQ3S probably better | | GGUF | i1-Q3KL | 4.4 | IQ3M probably better | | GGUF | i1-IQ4XS | 4.5 | | | GGUF | i1-Q40 | 4.8 | fast, low quality | | GGUF | i1-IQ4NL | 4.8 | prefer IQ4XS | | GGUF | i1-Q4KS | 4.8 | optimal size/speed/quality | | GGUF | i1-Q4KM | 5.0 | fast, recommended | | GGUF | i1-Q41 | 5.2 | | | GGUF | i1-Q5KS | 5.7 | | | GGUF | i1-Q5KM | 5.8 | | | GGUF | i1-Q6K | 6.7 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

weighted/imatrix quants of https://huggingface.co/summykai/gemma3-27b-abliterated-dpo For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/gemma3-27b-abliterated-dpo-GGUF This is a vision model - mmproj files (if any) will be in the static repository. Usage If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | GGUF | i1-IQ1S | 6.4 | for the desperate | | GGUF | i1-IQ1M | 6.9 | mostly desperate | | GGUF | i1-IQ2XXS | 7.8 | | | GGUF | i1-IQ2XS | 8.5 | | | GGUF | i1-IQ2S | 8.9 | | | GGUF | i1-IQ2M | 9.6 | | | GGUF | i1-Q2KS | 9.9 | very low quality | | GGUF | i1-Q2K | 10.6 | IQ3XXS probably better | | GGUF | i1-IQ3XXS | 10.8 | lower quality | | GGUF | i1-IQ3XS | 11.7 | | | GGUF | i1-IQ3S | 12.3 | beats Q3K | | GGUF | i1-Q3KS | 12.3 | IQ3XS probably better | | GGUF | i1-IQ3M | 12.6 | | | GGUF | i1-Q3KM | 13.5 | IQ3S probably better | | GGUF | i1-Q3KL | 14.6 | IQ3M probably better | | GGUF | i1-IQ4XS | 14.9 | | | GGUF | i1-Q40 | 15.7 | fast, low quality | | GGUF | i1-Q4KS | 15.8 | optimal size/speed/quality | | GGUF | i1-Q4KM | 16.6 | fast, recommended | | GGUF | i1-Q41 | 17.3 | | | GGUF | i1-Q5KS | 18.9 | | | GGUF | i1-Q5KM | 19.4 | | | GGUF | i1-Q6K | 22.3 | practically like static Q6K | Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better): And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See https://huggingface.co/mradermacher/modelrequests for some answers to questions you might have and/or if you want some other model quantized. I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.