NexesQuants
TeeZee_Kyllene-Yi-34B-v1.1-iMat.GGUF
Quants with iMatrix for : https://huggingface.co/TeeZee/Kyllene-34B-v1.1 Non-iMatrix quants (more choice in higher bitrate quants) : https://huggingface.co/TeeZee/Kyllene-34B-v1.1-GGUF/tree/main Full offload possible on 48GB VRAM with a huge context size : Q80 Full offload possible on 36 GB VRAM with a huge context size : Q5KS Full offload possible on 24GB VRAM with a big to huge context size (from 12288 with Q4KM, for example) : Q4KM, Q4KS, Q3KM Full offload possible on 16GB VRAM with a decent context size : IQ3XXS SOTA (which is equivalent to a Q3KS with more context!), Q2K, Q2KS Full offload possible on 12GB VRAM with a decent context size : IQ2XS SOTA. lower quality : IQ2XXS SOTA Full offload maybe possible on 8GB VRAM with a small context size : IQ1S revision "even better" (b2404) (or v5). All my IQ1S quant from the 13/03/2024 will be with this new IQ1S quantization base. The merge parameters and logs are in the repo : https://huggingface.co/TeeZee/Kyllene-34B-v1.1/tree/main After iMatrixing and quantizing Kyllene, I benched her thoroughly, and she proved herself worthy : Q4KS : - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,Hellaswag,85,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,Hellaswag,85.2,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,Hellaswag,84.6,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,HellaswagBin,81,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,HellaswagBin,83.5,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,HellaswagBin,82.95,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,Arc-Challenge,61.53846154,,299,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,Arc-Easy,80.35087719,,570,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,MMLU,43.13099042,,313,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,Thruthful-QA,35.00611995,,817,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,Winogrande,79.3212,,1267,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KS.gguf,-,wikitext,5.1703,512,512,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, Q4KM : - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,Hellaswag,84.75,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,Hellaswag,85.6,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,Hellaswag,84.9,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,HellaswagBin,81,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,HellaswagBin,83.4,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,HellaswagBin,82.9,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,Arc-Challenge,60.53511706,,299,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,Arc-Easy,80.52631579,,570,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,MMLU,42.49201278,,313,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,Thruthful-QA,34.39412485,,817,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,Winogrande,79.4791,,1267,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,wikitext,5.1679,512,512,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,wikitext,4.3623,4096,4096,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q4KM.gguf,-,wikitext,4.4061,8192,8192,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, Q5KS : - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,Hellaswag,85.25,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,Hellaswag,85.6,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,Hellaswag,84.95,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,HellaswagBin,81.25,,400,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,HellaswagBin,83.3,,1000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,HellaswagBin,83,,2000,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,Arc-Challenge,60.20066890,,299,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,Arc-Easy,81.05263158,,570,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,MMLU,42.17252396,,313,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,Thruthful-QA,36.96450428,,817,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,Winogrande,79.5580,,1267,2024-01-28 05:40:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - Kyllene-34B-v1.1-b1989-iMat-c32ch3250-Q5KS.gguf,-,wikitext,5.1806,512,512,2024-01-28 00:00:00,,34b,Yi,200000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,Hellaswag,70.3,,1000,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,Arc-Challenge,40.46822742,299,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,Arc-Easy,62.28070175,,570,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,MMLU,32.90734824,,313,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,Thruthful-QA,29.37576499,,817,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,Winogrande,68.7451,,1267,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,wikitext,9.8761,512,512,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex, - TeeZeeKyllene-34B-v1.1-b2409-iMat-c32ch3250-IQ1Sv5.gguf,-,wikitext,7.8954,4096,4096,2024-03-12 00:00:00,,34b,Yi,2000000,,,GGUF,TeeZee,Nexesenex,
MIstral-QUantized-70b_Miqu-1-70b-iMat.GGUF
Llama_3.2_3b_Kermes_v2.1-iMat-CQ-GGUF
zai-org_GLM-4.5-Air-bf16-iMat-IKL-CQ-GGUF
The IQ4XSL quant is meant to be run with IKLlama.cpp or the incoming EsoCroK.Cpp, because it uses a Q60 quant specific to IKLlama.cpp Here's EsoCrok, compatible with the llama.cpp mainline quants AND Q60 The usual Croco has not been updated (yet?) to support properly GLM 4.5 (or OpenAI GPT OSS)
huihui-ai_Llama-3.3-70B-Instruct-abliterated-bf16-iMat-CQ-GGUF
Dolphin3.0-Llama3.1-1B-abliterated-GGUF
google_gemma-3-27b-it-qat-q4_0-unquantized-iMat-NXS-GGUF
Senku-70b-iMat.GGUF
Steelskull_L3.3-MS-Nevoria-70b-bf16-iMat-CQ-GGUF
gghfez_Writer-Large-2411-iMat-IKL-CQ-GGUF
SicariusSicarii_Nano_Imp_1B-iMat-IKL-NXS-CQ-GGUF
WinterGoddess-1.4x-limarpv3-70B-L2-32k-Requant.GGUF
Mistral-Large-Instruct-2407-bf16-iMat-CQ-GGUF
alpindale_WizardLM-2-8x22B-bf16-iMat-CQ-GGUF
vigogne-33b-instruct-GGUF
Undi95_Miqu-70B-Alpaca-DPO-iMat.GGUF
llama-3-70B-Instruct-abliterated-CQ-GGUF
alchemonaut_QuartetAnemoi-70B-iMat.GGUF
DeepSeek-V2-Lite-Chat-Q8_0-GGUF
Nexesenex/DeepSeek-V2-Lite-Chat-Q80-GGUF This model was converted to GGUF format from `deepseek-ai/DeepSeek-V2-Lite-Chat` using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model. Use with llama.cpp Install llama.cpp through brew (works on Mac and Linux) Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well. Step 2: Move into the llama.cpp folder and build it with `LLAMACURL=1` flag along with other hardware-specific flags (for ex: LLAMACUDA=1 for Nvidia GPUs on Linux).
mlabonne_gemma-3-27b-it-abliterated-iMat-GGUF
lama-3.3-Tess-3-70B-TA_v0.60-iMat-CQ-GGUF
ChuckMcSneed_WinterGoddess-1.4x-70b-32k-iMat.GGUF
Meta-Llama-3.1-70b-instruct-bf16-iMat-CQ-GGUF
Llama_3.2_1b_Odyssea_V1-GGUF
Nexesenex/Llama3.21bOdysseaV1-GGUF IMPORTANT : These models are quantized with IKLlama.cpp, not Llama.cpp This model was converted to GGUF format from `Nexesenex/Llama3.21bOdysseaV1` using llama.cpp's fork IK Llama via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model. Use with llama.cpp (I never tested that way with IKLlama) Install llama.cpp through brew (works on Mac and Linux) Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well. -> necessary to use Croco. Step 1: Clone llama.cpp from GitHub. -> necessary to use Croco. Step 2: Move into the llama.cpp folder and build it with `LLAMACURL=1` flag along with other hardware-specific flags (for ex: LLAMACUDA=1 for Nvidia GPUs on Linux).
sophosympatheia_New-Dawn-Llama-3-70B-32K-v1.0-iMat-CQ-GGUF
huihui-ai_DeepSeek-R1-Distill-Llama-70B-abliterated-Q8_0-iMat-CQ-GGUF
DeepSeek-V2-Lite-Chat-Uncensored-Unbiased-Reasoner-Q8_0-GGUF
mistral-small-3.1-24b-instruct-2503-iMat-IKLQ-GGUF
IQ2KS, IQ4KS, IQ5KS are Gen 2 IQK Quants from Ikawrakow. They are faster (PP for sure, TG maybe) than the gen 1 IKQuants (IQ2K to IQ6K). They'll work with my last release of Croco.Cpp. Cuda Pascal or more recent GPU needed, I didn't adapt or compile for anything else.
MaziyarPanahi_calme-2.3-llama3.1-70b-f16-iMat-CQ-GGUF
DeepSeek-V2-Lite-Chat-Uncensored-Unbiased-Q8_0-GGUF
Gemma-3-4b_X-Ray-Abli_Linear_v1.01-iMat-IKL-NXS-CQ-GGUF
DeepSeek-V2-Lite-Chat-Uncensored-Q8_0-GGUF
Nexesenex/DeepSeek-V2-Lite-Chat-Uncensored-Q80-GGUF This model was converted to GGUF format from `nicoboss/DeepSeek-V2-Lite-Chat-Uncensored` using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model. Use with llama.cpp Install llama.cpp through brew (works on Mac and Linux) Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well. Step 2: Move into the llama.cpp folder and build it with `LLAMACURL=1` flag along with other hardware-specific flags (for ex: LLAMACUDA=1 for Nvidia GPUs on Linux).
c4ai-command-r-plus-08-2024-bf16-iMat-CQ-GGUF
Llama_3.x_70b_Hexagon_Purple_V3-iMat-CQ-GGUF
TheDrummer_Agatha-111B-v1_iMat-IKL-CQ-GGUF
pankajmathur_orca_mini_v9_6_1B-instruct-Abliterated-LPL-GGUF
Llama_3.x_70b_Hexagon_Purple_V2-iMat-CQ-GGUF
Meta-Llama-3-70B-Instruct.GGUF
Llama-3.3-Nemotron-70B-Instruct-Abliterated-TA_v0.10-iMat-CQ-GGUF
Llama_3.x_70b_Trojka_V3-iMat-CQ-GGUF
Llama_3.x_70b_Nemotron-L3.3_abliterated_fusion_v2-Mat-CQ-GGUF
TareksLab_L3.3-TRP-BASE-80-70B-iMat-CQ-GGUF
alicecomfy_miqu-70b-openhermes-full-iMat.GGUF
Nous-Hermes-2-Llama-2-70B-iMat.GGUF
cognitivecomputations_dolphin-2.9.1-llama-3-70b-iMat-CQ-GGUF
google_gemma-3-12b-it-qat-q4_0-unquantized-iMat-NXS-GGUF
MaziyarPanahi_calme-2.4-rys-78b-f16-iMat-CQ-GGUF
Llama_3.x_70b_Triads_V8-iMat-CQ-GGUF
huihui-ai_Llama-3_1-Nemotron-51B-Instruct-abliterated-iMat-CQ-GGUF
Llama_3.x_70b_Tristar_V1.0-iMat-CQ-GGUF
dfurman_CalmeRys-78B-Orpo-v0.1-bf16-iMat-CQ-GGUF
Fallen-Gemma3-27B-v1-Q8_0-iMat-IKL-NXS-GGUF
google_gemma-3-4b-it-qat-q4_0-unquantized-iMat-NXS-GGUF
L3-70B-daybreak-abliterated-v0.4-BF16-GGUF
Llama_3.x_70b_L3.3_Dolphin_128K_v1.02-iMat-CQ-GGUF
Llama_3.x_70b_L3.3-DoppelGutenberg_abliterated_fusion-iMat-CQ-GGUF
NeverSleep_MiquMaid-v2-70B-DPO-iMat.GGUF
alchemonaut_BoreanGale-70B-iMat.GGUF
NeverSleep_MiquMaid-v2-70B-iMat.GGUF
mistralai_Mixtral-8x22B-Instruct-v0.1-bf16-iMat-CQ-GGUF
Llama_3.x_70b_Trojka_V3.8-iMat-CQ-GGUF
EVA-UNIT-01_EVA-LLaMA-3.33-70B-v0.1-bf16-iMat-CQ-GGUF
Llama_3.3_70b_Evalseuses_v1.0-iMat-CQ-GGUF
MiquMaid-v2-70B-alpha-Requant-iMat.GGUF
TheDrummer_Behemoth-123B-v2-bf16-iMat-CQ-GGUF
migtissera_Tess-3-Llama-3.1-70B-iMat-CQ-GGUF
Llama_3.x_70b_FreeFaller_R1_V1.11-iMat-CQ-GGUF
Llama_3.x_70b_Athoblumicreacalmtess_v1.0-iMat-CQ-GGUF
nvidia_Llama-3_3-Nemotron-Super-49B-v1-GGUF
Sao10K_L3.1-70B-Euryale-v2.2-f16-iMat-GGUF
mlabonne_Hermes-3-Llama-3.1-70B-lorablated-bf16-iMat-CQ-GGUF
Sao10K_L3.1-70B-Hanami-x1-bf16-iMat-CQ-GGUF
Llama_3.x_70b_NegaTessTeaz_0.2-iMat-CQ-GGUF
TareksLab_DM1-B-iMat-CQ-GGUF
Llama_3.3_70b_DarkHorse-iMat-CQ-GGUF
Llama_3.3_70b_Wayfarer_Negative_fusion_v2-iMat-CQ-GGUF
SicariusSicariiStuff_Negative_LLAMA_70B-bf16-iMat-CQ-GGUF
Steelskull_L3.3-Damascus-R1-bf16-iMat-CQ-GGUF
Sao10K_L3-70B-Euryale-v2.1-bf16-iMat-CQ-GGUF
Llama_3.x_70b_Kraken_v1.1-iMat-CQ-GGUF
Llama_3.x_70b_Nemdohertess_v2.0-iMat-CQ-GGUF
Llama_3.x_70b_biofallevabigsafetess_v1.0-iMat-CQ-GGUF
hitachi-nlp_Llama-3.1-70B-FLDx2-iMat-CQ-GGUF
Llama_3.x_70b_SmarTricks_V1.30-iMat-CQ-GGUF
Llama_3.x_70b_DoppelGutenberg-L3.3_abliterated_fusion_v2-iMat-CQ-GGUF
NousResearch_Yarn-Llama-2-70b-32k-iMat.GGUF
Mixtral-8x22B-v0.1-iMat.GGUF
Sao10K_L3.1-70B-Euryale-v2.2-f16-iMat-CQ-GGUF
MC_Llama3.3-70B-Nemotron3.1-Eva0.1-ocypetp-bf16-iMat-CQ-GGUF
GGUF quant(s) for this model: https://huggingface.co/mergekit-community/mergekit-dareties-ocypetp The first quant published uses another Imatrix (Bartowski's iMatrix for Nevoria 70b, first of the name. But the model is good already for RP. Very good, actually. Edit : I made an iMatix for this model, and requanted-> v2). Sadly (or hopefully, it means that the iMatrix of a given model can be quite useful to quantize another with quite a good approximation) a dedicated imatrix doesn't fix the high perplexity. (+0.35-0.5 compared to most of L3.1/3 merges). I guess I didn't pick the most suitable merge, perplexity wise. The q50 quant is a direct conversion from the bf16 weights, converted with Q80 embeddings, output weight, and attnv, the rest in Q50, and I used that quant to make the iMatrix. Edit again : After testing, the high perplexity of this merge (4.5 for wikitext 512ctx instead of 3.9-4.1) is, at least partly, due to Eva 0.1 itself (which has +0.3 ppl wiki 512ctx compared to this merge). That means, considering its prose, that this merge retained the qualities of Eva 0.1 with a lower ppl, and the qualities of Nemotron 3.1 instruct as well, added to those of Llama 3.3 instruct used as a base, both in their non abliterated versions.
TheDrummer_Behemoth-123B-v1.2-bf16-iMat-CQ-GGUF
juvi21_Hermes-2-Theta-L3-Euryale-Ties-0.8-70B-bf16-iMat-CF.GGUF
Llama_3.x_70b_EvaTessTeaz_0.21-iMat-CQ-GGUF
Llama_3.1_70b_Luminemoblivion_V1.01-iMat-CQ-GGUF
Llama_3.x_70b_Triads_V5-iMat-CQ-GGUF
Llama_3.x_70b_NemeSlices_V1.5-iMat-CQ-GGUF
Llama_3.x_70b_TrinitEva_V1.1-iMat-CQ-GGUF
Llama_3.1_70b_Tesseire_V1-iMat-CQ-GGUF
Llama_3.x_70b_Flipper_0.34-iMat-CQ-GGUF
Llama_3.x_70b_Trojka_V1.5-iMat-CQ-GGUF
sophosympatheia_Electranova-70B-v1.0-iMat-CQ-GGUF
Llama_3.x_70b_SmarTricks_v1.61-iMat-CQ-GGUF
nvidia_Llama-3.1-Nemotron-70B-Instruct-bf16-iMat-CQ-GGUF
TheSkullery_L3.3-Exp-Nevoria-70b-v0.1-bf16-iMat-CQ.GGUF
TheDrummer_Behemoth-123B-v2.1-bf16-iMat-CQ-GGUF
TheSkullery_L3.3-Exp-Nevoria-R1-70b-v0.1-bf16-iMat-CQ-GGUF
TheDrummer_Behemoth-123B-v2.2-bf16-iMat-CQ-GGUF
Llama_3.x_70b_Nemeslices_V1.4-iMat-CQ-GGUF
Llama_3.x_70b_NegerTeaz_0.21-iMat-CQ-GGUF
Llama_3.3_70b_DeepSeek_R1_Dropable_V1.01-iMat-CQ-GGUF
Llama_3.1_70b_TearDrops_V1.11-iMat-CQ-GGUF
Llama_3.x_70b_BetsyTeaz_0.2-iMat-CQ-GGUF
Llama_3.x_70b_SmarTracks_V1.01-iMat-CQ-GGUF
Llama_3.x_70b_Tessessence_0.10f-iMat-CQ-GGUF
Llama_3.x_70b_Nemesis_V1.1-iMat-CQ-GGUF
gemma-3-27b-novision-Q8_0-GGUF
LLlama_3.x_70b_SmarTracks_V1.30_flat-iMat-CQ-GGUF
Llama_3.x_70b_Hexagon_Grey_v1.0-iMat-CQ-GGUF
Llama_3.3_70b_Negative_Wayfarer_fusion-v1-iMat-CQ-GGUF
Llama_3.x_70b_TDRussel-Storywriter_128K_Dop_v1.02-iMat-CQ-GGUF
miqu-1-70b-iMat-IKL-CQ-GGUF
Mistral-Large-Instruct-2411-bf16-iMat-CQ-GGUF
mlabonne_Llama-3.1-70B-Instruct-lorablated-bf16-iMat-CQ-GGUF
nbeerbower_Llama-3.1-Nemotron-lorablated-70B-bf16-iMat-CQ-GGUF
TheDrummer_Behemoth-123B-v1-bf16-iMat-CQ-GGUF
MC_Anubis-Llama3.3-70B-Nemotron3.1-Eva0.1-stjgmmc-bf16-iMat-CQ-GGUF
TheDrummer_Behemoth-123B-v1.1-bf16-iMat-CQ-GGUF
NaniDAO_Llama-3.3-70B-Instruct-ablated-iMat-CF-GGUF
huihui-ai_Llama-3.3-70B-Instruct-abliterated-finetuned-bf16-iMat-CQ-GGUF
Llama_3.x_70b_NegaTess_0.1-iMat-CQ-GGUF
Llama_3.2_1b_Synopsys_0.11-GGUF
Llama_3.2_1b_Odyssea_V1.01-GGUF
Llama_3.2_1b_Odyssea_V1.01a-GGUF
Llama_3.2_1b_Odyssea_V1.01b-GGUF
Nexesenex/Llama3.21bOdysseaV1.01b-GGUF IMPORTANT : These models are quantized with IKLlama.cpp, not Llama.cpp This model was converted to GGUF format from `Nexesenex/Llama3.21bOdysseaV1.01b` using llama.cpp's fork IK Llama via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model. Use with llama.cpp (I never tested that way with IKLlama) Install llama.cpp through brew (works on Mac and Linux) Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well. -> necessary to use Croco. Step 1: Clone llama.cpp from GitHub. -> necessary to use Croco. Step 2: Move into the llama.cpp folder and build it with `LLAMACURL=1` flag along with other hardware-specific flags (for ex: LLAMACUDA=1 for Nvidia GPUs on Linux).