OpenVINO

Model creator: BAAI Original model: bge-base-en-v1.5 Description This is bge-base-en-v1.5 model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.3.0 and higher Optimum Intel 1.25.2 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. You can find more detailed usage examples in OpenVINO Notebooks: The original model is distributed under MIT license. More details can be found in bge-base-en-v1.5. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-Reranker-0.6B-fp16-ov Model creator: Qwen Original model: Qwen3-Reranker-0.6B Description This is Qwen3-Reranker-0.6B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.4.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-Reranker-0.6B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Mistral-7B-Instruct-v0.3-int4-cw-ov Model creator: Mistral AI Original model: Mistral-7B-Instruct-v0.3 This is Mistral-7B-Instruct-v0.3 model converted to the OpenVINO™ IR (Intermediate Representation) format. > [!NOTE] > The model is optimized for inference on NPU using these instructions. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Intel® NPU Driver - Windows 32.0.100.4023 for Intel® Core™ Ultra processors and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-0.6B-fp16-ov Model creator: Qwen Original model: Qwen3-0.6B Description This is Qwen3-0.6B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-0.6B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-0.6B-int4-ov Model creator: Qwen Original model: Qwen3-0.6B Description This is Qwen3-0.6B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-0.6B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: Microsoft Original model: Phi-3.5-vision-instruct This is microsoft/Phi-3.5-vision-instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 using Activation Aware Quantization (AWQ) by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: mode: INT4ASYM ratio: 1.0 groupsize: 128 awq: True dataset: contextual numsamples: 32 The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.0.0 and higher Optimum Intel 1.21.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: The original model is distributed under MIT license. More details can be found in original model card.

Phi-3.5-mini-instruct-int4-cw-ov Model creator: microsoft Original model: Phi-3.5-mini-instruct Description This is Phi-3.5-mini-instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. > [!NOTE] > The model is optimized for inference on NPU using these instructions. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Intel® NPU Driver - Windows 32.0.100.4023 for Intel® Core™ Ultra processors and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under mit license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Mistral-7B-Instruct-v0.2-int4-cw-ov Model creator: Mistral AI Original model: Mistral-7B-Instruct-v0.2 This is Mistral-7B-Instruct-v0.2 model converted to the OpenVINO™ IR (Intermediate Representation) format. > [!NOTE] > The model is optimized for inference on NPU using these instructions. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Intel® NPU Driver - Windows 32.0.100.4023 for Intel® Core™ Ultra processors and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: OpenGVLab Original model: InternVL2-1B This is OpenGVLab/InternVL2-1B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: mode: INT8ASYM The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.0.0 and higher Optimum Intel 1.21.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under MIT license. More details can be found in original model card.

Qwen3-Embedding-0.6B-fp16-ov Model creator: Qwen Original model: Qwen3-Embedding-0.6B Description This is Qwen3-Embedding-0.6B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.4.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-Embedding-0.6B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

DeepSeek-R1-Distill-Qwen-1.5B-int4-cw-ov Model creator: DeepSeek Original model: DeepSeek-R1-Distill-Qwen-1.5B This is DeepSeek-R1-Distill-Qwen-1.5B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. > [!NOTE] > The model is optimized for inference on NPU using these instructions. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Intel® NPU Driver - Windows 32.0.100.4023 for Intel® Core™ Ultra processors and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under mit license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-1.7B-int4-ov Model creator: Qwen Original model: Qwen3-1.7B Description This is Qwen3-1.7B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-1.7B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: Microsoft Original model: Phi-3.5-vision-instruct This is microsoft/Phi-3.5-vision-instruct model converted to the OpenVINO™ IR (Intermediate Representation) format. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.0.0 and higher Optimum Intel 1.21.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: The original model is distributed under MIT license. More details can be found in original model card.

DeepSeek-R1-Distill-Qwen-7B-int4-cw-ov Model creator: DeepSeek Original model: DeepSeek-R1-Distill-Qwen-7B This is DeepSeek-R1-Distill-Qwen-7B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. > [!NOTE] > The model is optimized for inference on NPU using these instructions. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Intel® NPU Driver - Windows 32.0.100.4023 for Intel® Core™ Ultra processors and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under mit license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

gemma-2b-it-int4-ov Model creator: google Original model: gemma-2b-it Description This is gemma-2b-it model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2024.5.0 and higher Optimum Intel 1.21.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the OpenVINO Large Language Model Inference Guide. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under gemma license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-1.7B-fp16-ov Model creator: Qwen Original model: Qwen3-1.7B Description This is Qwen3-1.7B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-1.7B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

gpt-j-6b-int4-cw-ov Model creator: EleutherAI Original model: gpt-j-6b Description This is gpt-j-6b model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. > [!NOTE] > The model is optimized for inference on NPU using these instructions. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Intel® NPU Driver - Windows 32.0.100.4023 for Intel® Core™ Ultra processors and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-1.5B-Instruct-int4-ov Model creator: Qwen Original model: Qwen2.5-Coder-1.5B-Instruct Description This is Qwen2.5-Coder-1.5B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-1.5B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: mistral-community Original model: mistral-community/pixtral-12b This is mistral-community/pixtral-12b model converted to the OpenVINO™ IR (Intermediate Representation) format. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

gemma-2b-it-int8-ov Model creator: google Original model: gemma-2b-it Description This is gemma-2b-it model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2024.5.0 and higher Optimum Intel 1.21.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the OpenVINO Large Language Model Inference Guide. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under gemma license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

This is a pruned model, originating from the Qwen/Qwen3-0.6B. The model was built to accompany Qwen/Qwen3-8B and to be used as a draft model in the context of Speculative Decoding. The pruning was performed by applying the findings from recent layer-wise pruning research, see one of the relevant publications, followed by the accuracy recovery fine-tuning over synthetic data generated by the target model Qwen/Qwen3-8B. Qwen3-pruned-6L-from-0.6B-int8-ov is a model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to int8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: mode: INT8ASYM For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2 and higher Optimum Intel 1.25.3 and higher 1. Install packages required for using OpenVINO GenAI with Speculative decoding: 3. Run model inference using the speculative decoding and specify the pipeline parameters: More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The model is distributed under the Intel Research Use License Agreement. The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-0.6B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

gemma-7b-it-int4-ov Model creator: Google Original model: gemma-7b-it Description This is gemma-7b-it model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2024.4.0 and higher Optimum Intel 1.20.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the OpenVINO Large Language Model Inference Guide. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Limitations Check the original model card for original model card for limitations. The original model is distributed under gemma license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-1.7B-int8-ov Model creator: Qwen Original model: Qwen3-1.7B Description This is Qwen3-1.7B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-1.7B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-0.6B-int8-ov Model creator: Qwen Original model: Qwen3-0.6B Description This is Qwen3-0.6B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-0.6B. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

phi-4-int4-ov Model creator: microsoft Original model: phi-4 Description This is phi-4 model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under mit license. More details can be found in phi-4. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Phi-4-reasoning-fp16-ov Model creator: microsoft Original model: Phi-4-reasoning Description This is Phi-4-reasoning model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under mit license. More details can be found in Phi-4-reasoning. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Phi-4-reasoning-int8-ov Model creator: microsoft Original model: Phi-4-reasoning Description This is Phi-4-reasoning model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under mit license. More details can be found in Phi-4-reasoning. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Phi-4-reasoning-int4-ov Model creator: microsoft Original model: Phi-4-reasoning Description This is Phi-4-reasoning model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under mit license. More details can be found in Phi-4-reasoning. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: OpenGVLab Original model: InternVL2-1B This is OpenGVLab/InternVL2-1B model converted to the OpenVINO™ IR (Intermediate Representation) format. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.0.0 and higher Optimum Intel 1.21.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under MIT license. More details can be found in original model card.

phi-4-int8-ov Model creator: microsoft Original model: phi-4 Description This is phi-4 model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under mit license. More details can be found in phi-4. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-14B-Instruct-fp16-ov Model creator: Qwen Original model: Qwen2.5-14B-Instruct Description This is Qwen2.5-14B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: - LLM - RAG text generation - Convert models from ModelScope to OpenVINO The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-14B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-7B-Instruct-int8-ov Model creator: Qwen Original model: Qwen2.5-Coder-7B-Instruct Description This is Qwen2.5-Coder-7B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-7B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-1.5B-Instruct-fp16-ov Model creator: Qwen Original model: Qwen2.5-1.5B-Instruct Description This is Qwen2.5-1.5B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-1.5B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

phi-4-fp16-ov Model creator: microsoft Original model: phi-4 Description This is phi-4 model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.1.0 and higher Optimum Intel 1.24.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under mit license. More details can be found in phi-4. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-0.5B-Instruct-int8-ov Model creator: Qwen Original model: Qwen2.5-Coder-0.5B-Instruct Description This is Qwen2.5-Coder-0.5B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-0.5B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-1.5B-Instruct-int8-ov Model creator: Qwen Original model: Qwen2.5-Coder-1.5B-Instruct Description This is Qwen2.5-Coder-1.5B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-1.5B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: Qwen Original model: Qwen/Qwen2-VL-7B-Instruct This is Qwen/Qwen2-VL-7B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: Qwen Original model: Qwen/Qwen2-VL-7B-Instruct This is Qwen/Qwen2-VL-7B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-0.5B-Instruct-int4-ov Model creator: Qwen Original model: Qwen2.5-Coder-0.5B-Instruct Description This is Qwen2.5-Coder-0.5B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-0.5B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: llava-hf Original model: llava-v1.6-mistral-7b-hf This is llava-hf/llava-v1.6-mistral-7b-hf model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: The provided OpenVINO™ IR model is compatible with: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Mistral-7B-Instruct-v0.3-int4-ov Model creator: Mistral AI Original model: Mistral-7B-Instruct-v0.3 This is Mistral-7B-Instruct-v0.3 model converted to the OpenVINO™ IR (Intermediate Representation) format. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: mistral-community Original model: mistral-community/pixtral-12b This is mistral-community/pixtral-12b model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-1.5B-Instruct-fp16-ov Model creator: Qwen Original model: Qwen2.5-Coder-1.5B-Instruct Description This is Qwen2.5-Coder-1.5B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-1.5B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-3B-Instruct-fp16-ov Model creator: Qwen Original model: Qwen2.5-Coder-3B-Instruct Description This is Qwen2.5-Coder-3B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-3B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-3B-Instruct-int8-ov Model creator: Qwen Original model: Qwen2.5-Coder-3B-Instruct Description This is Qwen2.5-Coder-3B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-3B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Mistral-7B-Instruct-v0.3-int8-ov Model creator: Mistral AI Original model: Mistral-7B-Instruct-v0.3 This is Mistral-7B-Instruct-v0.3 model converted to the OpenVINO™ IR (Intermediate Representation) format. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen2.5-Coder-0.5B-Instruct-fp16-ov Model creator: Qwen Original model: Qwen2.5-Coder-0.5B-Instruct Description This is Qwen2.5-Coder-0.5B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.25.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples You can find more detaild usage examples in OpenVINO Notebooks: The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen2.5-Coder-0.5B-Instruct. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Qwen3-Reranker-0.6B-seq-cls-fp16-ov Model creator: tomaarsen Original model: Qwen3-Reranker-0.6B-seq-cls Description This is Qwen3-Reranker-0.6B-seq-cls model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to FP16. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.4.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the Inference with Optimum Intel. The original model is distributed under Apache License Version 2.0 license. More details can be found in Qwen3-Reranker-0.6B-seq-cls. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

gemma-7b-fp16-ov Model creator: Google Original model: gemma-7b The provided OpenVINO™ IR model is compatible with: OpenVINO version 2024.4.0 and higher Optimum Intel 1.20.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: For more examples and possible optimizations, refer to the OpenVINO Large Language Model Inference Guide. 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under gemma license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: llava-hf Original model: llava-v1.6-mistral-7b-hf This is llava-hf/llava-v1.6-mistral-7b-hf model converted to the OpenVINO™ IR (Intermediate Representation) format. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: mistral-community Original model: mistral-community/pixtral-12b This is mistral-community/pixtral-12b model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: Qwen Original model: Qwen/Qwen2-VL-7B-Instruct This is Qwen/Qwen2-VL-7B-Instruct model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

distil-small.en-int8-ov Model creator: Distil-whisper Original model: distil-small.en Description This is distil-small.en model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.23.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under mit license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

distil-small.en-int4-ov Model creator: Distil-whisper Original model: distil-small.en Description This is distil-small.en model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.23.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under mit license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

distil-medium.en-int8-ov Model creator: Distil-whisper Original model: distil-medium.en Description This is distil-medium.en model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.23.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under mit license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

distil-medium.en-int4-ov Model creator: Distil-whisper Original model: distil-medium.en Description This is distil-medium.en model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.23.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples Check the original model card for original model card for limitations. The original model is distributed under mit license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Mistral-7B-Instruct-v0.3-fp16-ov Model creator: Mistral AI Original model: Mistral-7B-Instruct-v0.3 This is Mistral-7B-Instruct-v0.3 model converted to the OpenVINO™ IR (Intermediate Representation) format. The provided OpenVINO™ IR model is compatible with: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: OpenGVLab Original model: InternVL2-8B This is OpenGVLab/InternVL2-8B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 using Activation Aware Quantization (AWQ) by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: mode: INT4ASYM ratio: 1.0 groupsize: 128 awq: True dataset: contextual numsamples: 32 The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under MIT license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: llava-hf Original model: llava-v1.6-mistral-7b-hf This is llava-hf/llava-v1.6-mistral-7b-hf model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT8 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.2.0 and higher Optimum Intel 1.26.0 and higher 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under apache-2.0 license. More details can be found in original model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model creator: Black Forset Labs Original model: black-forest-labs/FLUX.1-schnell This is black-forest-labs/FLUX.1-schnell model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF. Weight compression was performed using `nncf.compressweights` with the following parameters: For more information on quantization, check the OpenVINO model optimization guide. The provided OpenVINO™ IR model is compatible with: OpenVINO version 2025.0.0 and higher Optimum Intel 1.22.0 and higher 1. Install packages required for using Optimum Intel integration with the OpenVINO backend: 1. Install packages required for using OpenVINO GenAI. More GenAI usage examples can be found in OpenVINO GenAI library docs and samples The original model is distributed under Apache 2.0 license. More details can be found in model card. Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.