noctrex

Original model: https://huggingface.co/datalab-to/chandra Try to use the best quality you can run. For the mmproj, try to use the F32 version as it will produce the best results. F32 > BF16 > F16

This are the imatrix quantizations of the model LightOnOCR-1B-1025 Original model: https://huggingface.co/lightonai/LightOnOCR-1B-1025 Try to use the best quality you can run. For the mmproj, try to use the F32 version as it will produce the best results. F32 > BF16 > F16

This is a MXFP4MOE quantization of the model GLM-4.5-Air-REAP-82B-A12B Original model: https://huggingface.co/cerebras/GLM-4.5-Air-REAP-82B-A12B

These are quantizations of the model Huihui-Mistral-Small-3.2-24B-Instruct-2506-abliterated-v2 Original model: https://huggingface.co/huihui-ai/Huihui-Mistral-Small-3.2-24B-Instruct-2506-abliterated-v2

This is a MXFP4MOE quantization of the model Qwen3-Coder-REAP-25B-A3B Added an imatrix version, based on the imatrix from bartowski. I also created my own imatrix versions, which are marked as codetiny-exp and codemedium-exp. This is considered experimental. What I did, is that I took a very specific dataset, that is ONLY for coding and not for general knowledge. It's codetiny dataset from eaddario/imatrix-calibration And codemedium dataset from eaddario/imatrix-calibration I thought that would be better suited, as this a coding specific model. But further tests must be done. Please provide feedback. Original model: https://huggingface.co/cerebras/Qwen3-Coder-REAP-25B-A3B

This is a MXFP4MOE quantization of the model MiniMax-M2 Original model: https://huggingface.co/unsloth/MiniMax-M2 It seems that the original model I quantized had chat template problems, so I re-quantized the unsloth version of it that has template fixes. Please delete the old one and download the new quant.

These are quantizations of the model Gelato-30B-A3B The imatrix has been used from mradermacher. As most of the quants are available from the great mradermacher team, I will include here only the quants that are missing. Usage Notes: - Download the latest llama.cpp to use these quantizations. - Try to use the best quality you can run. - For the `mmproj` file, the F32 version is recommended for best results (F32 > BF16 > F16).

This is a MXFP4MOE quantization of the model Huihui-gpt-oss-120b-BF16-abliterated-v2 Original model: https://huggingface.co/huihui-ai/Huihui-gpt-oss-120b-BF16-abliterated-v2

This is a MXFP4MOE quantization of the model Qwen3-VL-235B-A22B-Instruct Original model: https://huggingface.co/unsloth/Qwen3-VL-235B-A22B-Instruct

This is a MXFP4MOE quantization of the model Qwen3-VL-235B-A22B-Thinking Original model: https://huggingface.co/unsloth/Qwen3-VL-235B-A22B-Thinking-1M This is the version from unloth that has expanded the context size from 256k to 1M.

This are the quantizations of the model LightOnOCR-1B-1025 Original model: https://huggingface.co/lightonai/LightOnOCR-1B-1025 Try to use the best quality you can run. For the mmproj, try to use the F32 version as it will produce the best results. F32 > BF16 > F16

This is a MXFP4MOE quantization of the model Huihui-gpt-oss-20b-BF16-abliterated-v2 Original model: https://huggingface.co/huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated-v2

These are quantizations of the model Huihui-Qwen3-VL-2B-Instruct-abliterated. These quantizations were created using an imatrix merged from combined\all\large and harmful.txt to leverage the abliterated nature of the model. Usage Notes: - Download the latest llama.cpp to use these quantizations. - Try to use the best quality you can run. - For the `mmproj` file, the F32 version is recommended for best results (F32 > BF16 > F16).

This is a MXFP4MOE quantization of the model Tongyi-DeepResearch-30B-A3B Original model: https://huggingface.co/Alibaba-NLP/Tongyi-DeepResearch-30B-A3B

This is a MXFP4MOE quantization of the model cogito-v2-preview-llama-109B-MoE Model quantized with BF16 GGUF's from: https://huggingface.co/unsloth/cogito-v2-preview-llama-109B-MoE-GGUF Original model: https://huggingface.co/deepcogito/cogito-v2-preview-llama-109B-MoE

This is a MXFP4MOE quantization of the model DeepSeek-MoE-16B-Chat Original model: https://huggingface.co/deepseek-ai/deepseek-moe-16b-chat

This is a MXFP4MOE quantization of the model Qwen3-30B-A3B-CoderThinking-YOYO-linear Original model: https://huggingface.co/YOYO-AI/Qwen3-30B-A3B-CoderThinking-YOYO-linear

This is a MXFP4MOE quantization of the model Ling-flash-2.0 Original model: https://huggingface.co/inclusionAI/Ling-flash-2.0

This is a MXFP4MOE quantization of the model Kimi-VL-A3B-Thinking-2506 Original model: https://huggingface.co/moonshotai/Kimi-VL-A3B-Thinking-2506

This is a MXFP4 quant of Qwen3-Next-80B-A3B-Instruct The context has been extended from 256k to 1M, with YaRN as seen on the repo To enable it, run llama.cpp with options like: `--ctx-size 0 --rope-scaling yarn --rope-scale 4` ctx-size 0 sets it to 1M context, else set a smaller number like 524288 for 512k You can use also as normal if you don't want the extended context.

This is a MXFP4MOE quantization of the model Huihui-Tongyi-DeepResearch-30B-A3B-abliterated Original model: https://huggingface.co/huihui-ai/Huihui-Tongyi-DeepResearch-30B-A3B-abliterated

This is a MXFP4 quant of Qwen3-Next-80B-A3B-Thinking The context has been extended from 256k to 1M, with YaRN as seen on the repo To enable it, run llama.cpp with options like: `--ctx-size 0 --rope-scaling yarn --rope-scale 4` ctx-size 0 sets it to 1M context, else set a smaller number like 524288 for 512k You can use also as normal if you don't want the extended context.

This is a MXFP4MOE quantization of the model Llama-4-Scout-17B-16E-Instruct Model quantized with BF16 GGUF's from: https://huggingface.co/unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF Original model: https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct

This is a MXFP4MOE quantization of the model Qwen3-30B-A3B-YOYO-V4 Original model: https://huggingface.co/YOYO-AI/Qwen3-30B-A3B-YOYO-V4

This is a MXFP4MOE quantization of the model Qwen3-Coder-30B-A3B-Instruct-1M Original model: https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-GGUF Also added an imatrix version, based on the imatrix from unsloth. This is the version from unloth that has expanded the context size from 256k to 1M.

This is a MXFP4MOE quantization of the model Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B Original model: https://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B

These are quantizations of the model The-Philosopher-Zephyr-7B Original model: https://huggingface.co/Hypersniper/ThePhilosopherZephyr7B It's an older model back from 2023, based on the older Mistral 7B. Why quantize it now in 2025? Its so old! Well, I'm experimenting with importance matrices, and I used the text\en\large set stitched together with various philosophical stuff. Turns out it's quite fun!

This is a MXFP4MOE quantization of the model GLM-Steam-106B-A12B-v1 Original model: https://huggingface.co/TheDrummer/GLM-Steam-106B-A12B-v1

This is a MXFP4MOE quantization of the model ERNIE-4.5-21B-A3B-PT Model quantized with BF16 GGUF's from: https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-PT-GGUF Original model: https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-PT

This is a MXFP4MOE quantization of the model LLaDA-MoE-7B-A1B-Instruct-TD: A specialized instruction-tuned model, further optimized for accelerated inference using Trajectory Distillation. Also created a quant with an imatrix from mradermacher. Original model: https://huggingface.co/inclusionAI/LLaDA-MoE-7B-A1B-Instruct-TD

This is a MXFP4MOE quantization of the model Qwen3-30B-A3B-Mixture-2507 Original model: https://huggingface.co/YOYO-AI/Qwen3-30B-A3B-Mixture-2507

This is a MXFP4MOE quantization of the model Huihui-MoE-60B-A3B-abliterated Model quantized with F16 GGUF’s from: https://huggingface.co/DevQuasar/huihui-ai.Huihui-MoE-60B-A3B-abliterated-GGUF Original model: https://huggingface.co/DevQuasar/huihui-ai.Huihui-MoE-60B-A3B-abliterated-GGUF

These are quantizations of the model Qwen3-VL-30B-A3B-Instruct Original model: https://huggingface.co/unsloth/Qwen3-VL-30B-A3B-Instruct This is the 1M context length variant from unsloth, with their imatrix applied to it.

This is a MXFP4MOE quantization of the model Mixtral-8x7B-Instruct-v0.1 Original model: https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1

This is a MXFP4MOE quantization of the model Huihui-Granite-4.0-H-Micro-abliterated Original model: https://huggingface.co/huihui-ai/Huihui-granite-4.0-h-Micro-abliterated

This is a MXFP4MOE quantization of the model Qwen3-Yoyo-V4-42B-A3B-Thinking-TOTAL-RECALL Original model: https://huggingface.co/DavidAU/Qwen3-Yoyo-V4-42B-A3B-Thinking-TOTAL-RECALL

This is a MXFP4MOE quantization of the model PromptCoT-2.0-SelfPlay-30B-A3B Original model: https://huggingface.co/xl-zhao/PromptCoT-2.0-SelfPlay-30B-A3B

These are quantizations of the model Qwen3-VL-30B-A3B-Thinking Original model: https://huggingface.co/unsloth/Qwen3-VL-30B-A3B-Thinking This is the 1M context length variant from unsloth, with their imatrix applied to it.

This is a MXFP4MOE quantization of the model dolphin-2.7-mixtral-8x7b Original model: https://huggingface.co/dphn/dolphin-2.7-mixtral-8x7b

This is a MXFP4MOE quantization of the model GroveMoE-Inst Original model: https://huggingface.co/inclusionAI/GroveMoE-Inst

This is a MXFP4MOE quantization of the model Ling-Mini-2.0-Identity Original model: https://huggingface.co/qingy2024/Ling-Mini-2.0-Identity

This is a MXFP4MOE quantization of the model aquif-3.5-A4B-Think Original model: https://huggingface.co/aquif-ai/aquif-3.5-A4B-Think

This is a MXFP4MOE quantization of the model Ling-lite-1.5-2507 Original model: https://huggingface.co/inclusionAI/Ling-lite-1.5-2507

This is a MXFP4MOE quantization of the model SmallThinker-4BA0.6B-Instruct A quantization with imatrix is also included Original model: https://huggingface.co/PowerInfer/SmallThinker-4BA0.6B-Instruct

This is a MXFP4MOE quantization of the model Phi-mini-MoE-instruct Model quantized with F16 GGUF's from: https://huggingface.co/gabriellarson/Phi-mini-MoE-instruct-GGUF Original model: https://huggingface.co/microsoft/Phi-mini-MoE-instruct

This is a MXFP4MOE quantization of the model aquif-3-moe-17B-A2.8B-Think Original model: https://huggingface.co/aquif-ai/aquif-3-moe-17B-A2.8B-Think

This is a MXFP4MOE quantization of the model LLaDA-MoE-7B-A1B-Instruct Original model: https://huggingface.co/inclusionAI/LLaDA-MoE-7B-A1B-Instruct

This is a MXFP4MOE quantization of the model ERNIE-4.5-21B-A3B-Thinking Model quantized with BF16 GGUF's from: https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF Original model: https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-Thinking

This is a MXFP4MOE quantization of the model Moonlight-16B-A3B-Instruct Original model: https://huggingface.co/moonshotai/Moonlight-16B-A3B-Instruct

This is a MXFP4MOE quantization of the model Huihui-MoE-4.8B-A1.7B-abliterated Model quantized with F16 GGUF's from: https://huggingface.co/DevQuasar/huihui-ai.Huihui-MoE-4.8B-A1.7B-abliterated-GGUF Original model: https://huggingface.co/huihui-ai/Huihui-MoE-4.8B-A1.7B-abliterated

This is a MXFP4MOE quantization of the model Pristine-8B-A1B Also a imatrix quantization is included Original model: https://huggingface.co/qingy2024/Pristine-8B-A1B

This is a MXFP4MOE quantization of the model Huihui-MoE-12B-A4B-abliterated Original model: https://huggingface.co/huihui-ai/Huihui-MoE-12B-A4B-abliterated

This is a MXFP4MOE quantization of the model OLMoE-1B-7B-0125-Instruct Model quantized with F16 GGUF’s from: https://huggingface.co/DevQuasar/allenai.OLMoE-1B-7B-0125-Instruct-GGUF Original model: https://huggingface.co/allenai/OLMoE-1B-7B-0125-Instruct

This is a MXFP4MOE quantization of the model ERNIE-4.5-300B-A47B-PT Original model: https://huggingface.co/baidu/ERNIE-4.5-300B-A47B-PT This model's GGUF's have been removed, in order to conserve my repos use of space. If you want it, just message me, and I will make it available on demand.

This is a MXFP4MOE quantization of the model Phi-3.5-MoE-instruct Original model: https://huggingface.co/microsoft/Phi-3.5-MoE-instruct

This is a MXFP4MOE quantization of the model DeepSeek-V3.1-Terminus Model quantized with BF16 GGUF's from: https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF Original model: https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus

This is a MXFP4MOE quantization of the model Intern-S1 Original model: https://huggingface.co/internlm/Intern-S1