KaLM-embedding-multilingual-mini-instruct-v2.5
KaLM-Embedding-V2.5 is a versatile and compact embedding model, which achieves SOTA performance among models of comparable size and competes with models 3–26x larger by leveraging superior training techniques and data.
Model Details - Model Size: 0.5B - Embedding Dimension: 896 - Max Input Tokens: 32k - MRL dimensions: 896, 512, 256, 128, and 64 - Attn: Bidirectional attention - Pooling: Mean pooling
Training Recipe - Large-scale weakly supervised pretraining - High-quality supervised finetuning - Contrastive distillation with fine-grained soft labels
Additionally, focal-style sample reweighting and online hard-negative mixing are employed to emphasize difficult samples and enrich hard negatives.
- [x] Model Checkpoint - [x] KaLM-embedding-multilingual-mini-v1 - [x] KaLM-embedding-multilingual-mini-instruct-v1 - [x] KaLM-embedding-multilingual-mini-instruct-v1.5 - [x] KaLM-embedding-multilingual-mini-instruct-v2 - [x] KaLM-embedding-multilingual-mini-instruct-v2.5 - [x] KaLM-Embedding-Gemma3-12B-2511 - [x] Training and Evaluation Code: HITsz-TMG/KaLM-Embedding - [x] Technical Report: KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model - [x] Pre-training Data: Pre-training Data - [x] Fine-tuning Data: Fine-tuning Data
Evaluation Overall results on MTEB (cmn, v1) and MTEB (eng, v1).
OOD evaluation: KaLM-Embedding-V2.5 exhibits strong OOD generalization, competing with the 15x larger model in real-world retrieval scenarios.
Matryoshka embedding evaluation: KaLM-Embedding-V2.5 maintains robust performance with matryoshka embeddings even at smaller dimensions.
Requirements Since we have used the Qwen2 model, we advise you to install `transformers>=4.37.0`, or you might encounter the following error:
Usage sentence-transformers support Using this model becomes easy when you have sentence-transformers installed:
We add task instructions for asymmetric tasks: retrieval, reranking, classification, and clustering. And, we add task instructions for both queries and passages in symmetric tasks, including STS and pair classification. If you want to add task instructions to the query, you can use the model like this:
Or you can use `encodequery` and `encodedocument` to automatically add the default prompt for queries (`"Instruct: Given a query, retrieve documents that answer the query \n Query: "`) and documents (`""`), respectively.
Citation If you find this model useful, please consider giving a star and citation.
Contact If you encounter any issue, feel free to contact us via the email: ,