K-EXAONE-236B-A23B-GGUF

3.2K
14
—
by
LGAI-EXAONE
Language Model
OTHER
236B params
New
3K downloads
Early-stage
Try on Hugging FaceAdd to Compare
Edge AI:
Mobile
Laptop
Server
528GB+ RAM
Mobile
Laptop
Server
Quick Summary

AI model with specialized capabilities.

Device Compatibility

Mobile
4-6GB RAM
Laptop
16GB RAM
Server
GPU
Minimum Recommended
220GB+ RAM

Code Examples

Quickstartbash
# Download GGUF model weights (e.g. Q4_K_M)
hf download LGAI-EXAONE/K-EXAONE-236B-A23B-GGUF --include "*Q4_K_M*" --local-dir .

# Or convert huggingface model into GGUF format on your own
hf download LGAI-EXAONE/K-EXAONE-236B-A23B --local-dir $YOUR_MODEL_DIR
python convert_hf_to_gguf.py $YOUR_MODEL_DIR --outtype bf16 --outfile K-EXAONE-236B-A23B-BF16.gguf

# If you want to use the lower precision than BF16, you need to quantize the model
./llama-quantize K-EXAONE-236B-A23B-BF16.gguf K-EXAONE-236B-A23B-Q4_K_M.gguf Q4_K_M
bash
./llama-cli -m K-EXAONE-236B-A23B-Q4_K_M.gguf \
    -ngl 99 \
    -fa on -sm row \
    --temp 1.0 --top-p 0.95 --min-p 0 \
    -c 131072 -n 32768 \
    --no-context-shift \
    --jinja
bash
./llama-server -m K-EXAONE-236B-A23B-Q4_K_M.gguf \
    -ngl 99 \
    -fa on -sm row \
    --temp 1.0 --top-p 0.95 --min-p 0 \
    -c 131072 -n 32768 \
    --no-context-shift \
    --jinja \
    --host 0.0.0.0 --port 8080

Deploy This Model

Production-ready deployment in minutes

Together.ai

Instant API access to this model

Fastest API

Production-ready inference API. Start free, scale to millions.

Try Free API

Replicate

One-click model deployment

Easiest Setup

Run models in the cloud with simple API. No DevOps required.

Deploy Now

Disclosure: We may earn a commission from these partners. This helps keep LLMYourWay free.