Qwen3-30B-A3B-Thinking-2507-int4-AutoRound

97
8
30.0B
license:apache-2.0
by
Intel
Other
OTHER
30B params
New
97 downloads
Early-stage
Try on Hugging FaceAdd to Compare
Edge AI:
Mobile
Laptop
Server
68GB+ RAM
Mobile
Laptop
Server
Quick Summary

This model is an int4 model with groupsize 128 and symmetric quantization of Qwen/Qwen3-30B-A3B-Thinking-2507 generated by intel/auto-round algorithm.

Device Compatibility

Mobile
4-6GB RAM
Laptop
16GB RAM
Server
GPU
Minimum Recommended
28GB+ RAM

Code Examples

key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818
key dependency versiontextvllm
# key dependency version
torch                             2.8.0
transformers                      4.56.1
lm_eval                           0.4.9.1
vllm                              0.10.2rc3.dev106+g31bb760eb.precompiled
# vllm need to apply this pr https://github.com/vllm-project/vllm/pull/24818

Deploy This Model

Production-ready deployment in minutes

Together.ai

Instant API access to this model

Fastest API

Production-ready inference API. Start free, scale to millions.

Try Free API

Replicate

One-click model deployment

Easiest Setup

Run models in the cloud with simple API. No DevOps required.

Deploy Now

Disclosure: We may earn a commission from these partners. This helps keep LLMYourWay free.