TianchengGu

UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning Tiancheng Gu , Kaicheng Yang , Kaichen Zhang , Xiang An , Ziyong Feng, \ Yueyi Zhang , Weidong Cai , Jiankang Deng , Lidong Bing [](https://garygutc.github.io/UniME-v2/) []() [](https://github.com/GaryGuTC/UniME-v2) 💡 Highlights - We introduce an MLLM-as-a-Judge pipeline for hard negative mining that uses the advanced understanding capabilities of MLLM to assess the semantic alignment of each query-candidate pair within a globally retrieved potential hard negative set. - We present UniME-V2, a novel universal multimodal embedding model trained with an MLLM judgment based distribution alignment framework. By leveraging semantic matching scores as soft labels, the model effectively captures semantic differences between candidates, significantly enhancing its discriminative capability. Meanwhile, we propose UniME-V2-Reranker, a reranking model trained on high-quality, diverse hard negatives through a joint pairwise and listwise optimization approach. 💬 Support | Team Member | Email | |-------------|-------| | Tiancheng Gu | [](mailto:[email protected]) | | Kaicheng Yang | [](mailto:[email protected]) | 🖊️ Citation If you find this repository useful, please use the following BibTeX entry for citation. ⭐ Don't forget to star this repository if you find it helpful!

UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning Tiancheng Gu , Kaicheng Yang , Kaichen Zhang , Xiang An , Ziyong Feng, \ Yueyi Zhang , Weidong Cai , Jiankang Deng , Lidong Bing [](https://garygutc.github.io/UniME-v2/) []() [](https://github.com/GaryGuTC/UniME-v2) 💡 Highlights - We introduce an MLLM-as-a-Judge pipeline for hard negative mining that uses the advanced understanding capabilities of MLLM to assess the semantic alignment of each query-candidate pair within a globally retrieved potential hard negative set. - We present UniME-V2, a novel universal multimodal embedding model trained with an MLLM judgment based distribution alignment framework. By leveraging semantic matching scores as soft labels, the model effectively captures semantic differences between candidates, significantly enhancing its discriminative capability. Meanwhile, we propose UniME-V2-Reranker, a reranking model trained on high-quality, diverse hard negatives through a joint pairwise and listwise optimization approach. 💬 Support | Team Member | Email | |-------------|-------| | Tiancheng Gu | [](mailto:[email protected]) | | Kaicheng Yang | [](mailto:[email protected]) | 🖊️ Citation If you find this repository useful, please use the following BibTeX entry for citation. ⭐ Don't forget to star this repository if you find it helpful!

UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning Tiancheng Gu , Kaicheng Yang , Kaichen Zhang , Xiang An , Ziyong Feng, \ Yueyi Zhang , Weidong Cai , Jiankang Deng , Lidong Bing [](https://garygutc.github.io/UniME-v2/) []() [](https://github.com/GaryGuTC/UniME-v2) 💡 Highlights - We introduce an MLLM-as-a-Judge pipeline for hard negative mining that uses the advanced understanding capabilities of MLLM to assess the semantic alignment of each query-candidate pair within a globally retrieved potential hard negative set. - We present UniME-V2, a novel universal multimodal embedding model trained with an MLLM judgment based distribution alignment framework. By leveraging semantic matching scores as soft labels, the model effectively captures semantic differences between candidates, significantly enhancing its discriminative capability. Meanwhile, we propose UniME-V2-Reranker, a reranking model trained on high-quality, diverse hard negatives through a joint pairwise and listwise optimization approach. 💬 Support | Team Member | Email | |-------------|-------| | Tiancheng Gu | [](mailto:[email protected]) | | Kaicheng Yang | [](mailto:[email protected]) | 🖊️ Citation If you find this repository useful, please use the following BibTeX entry for citation. ⭐ Don't forget to star this repository if you find it helpful!

UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning Tiancheng Gu , Kaicheng Yang , Kaichen Zhang , Xiang An , Ziyong Feng, \ Yueyi Zhang , Weidong Cai , Jiankang Deng , Lidong Bing [](https://garygutc.github.io/UniME-v2/) []() [](https://github.com/GaryGuTC/UniME-v2) 💡 Highlights - We introduce an MLLM-as-a-Judge pipeline for hard negative mining that uses the advanced understanding capabilities of MLLM to assess the semantic alignment of each query-candidate pair within a globally retrieved potential hard negative set. - We present UniME-V2, a novel universal multimodal embedding model trained with an MLLM judgment based distribution alignment framework. By leveraging semantic matching scores as soft labels, the model effectively captures semantic differences between candidates, significantly enhancing its discriminative capability. Meanwhile, we propose UniME-V2-Reranker, a reranking model trained on high-quality, diverse hard negatives through a joint pairwise and listwise optimization approach. 💬 Support | Team Member | Email | |-------------|-------| | Tiancheng Gu | [](mailto:[email protected]) | | Kaicheng Yang | [](mailto:[email protected]) | 🖊️ Citation If you find this repository useful, please use the following BibTeX entry for citation. ⭐ Don't forget to star this repository if you find it helpful!