openmmlab-community

MM Grounding DINO model was proposed in An Open and Comprehensive Pipeline for Unified Object Grounding and Detection by Xiangyu Zhao, Yicheng Chen, Shilin Xu, Xiangtai Li, Xinjiang Wang, Yining Li, Haian Huang. MM Grounding DINO improves upon the Grounding DINO by improving the contrastive class head and removing the parameter sharing in the decoder, improving zero-shot detection performance on both COCO (50.6(+2.2) AP) and LVIS (31.9(+11.8) val AP and 41.4(+12.6) minival AP). You can find all the original MM Grounding DINO checkpoints under the MM Grounding DINO collection. You can use the raw model for zero-shot object detection. Here's how to use the model for zero-shot object detection: This model was trained on: - Objects365v1 - Open Images v6 - GOLD-G - V3Det - COCO 2017 - LVIS v1.0 - COCO 2014 - GRIT - RefCOCO - RefCOCO+ - RefCOCOg - gRefCOCO - Here's a table of models and their object detection performance results on COCO (results from official repo): | Model | Backbone | Pre-Train Data | Style | COCO mAP | | ------------------------------------------------------------------------------------------------------------------------------ | -------- | ------------------------ | --------- | ---------- | | mmgroundingdinotinyo365v1goldg | Swin-T | O365,GoldG | Zero-shot | 50.4(+2.3) | | mmgroundingdinotinyo365v1goldggrit | Swin-T | O365,GoldG,GRIT | Zero-shot | 50.5(+2.1) | | mmgroundingdinotinyo365v1goldgv3det | Swin-T | O365,GoldG,V3Det | Zero-shot | 50.6(+2.2) | | mmgroundingdinotinyo365v1goldggritv3det | Swin-T | O365,GoldG,GRIT,V3Det | Zero-shot | 50.4(+2.0) | | mmgroundingdinobaseo365v1goldgv3det | Swin-B | O365,GoldG,V3Det | Zero-shot | 52.5 | | mmgroundingdinobaseall | Swin-B | O365,ALL | - | 59.5 | | mmgroundingdinolargeo365v2oiv6goldg | Swin-L | O365V2,OpenImageV6,GoldG | Zero-shot | 53.0 | | mmgroundingdinolargeall | Swin-L | O365V2,OpenImageV6,ALL | - | 60.3 | - Here's a table of MM Grounding DINO tiny models and their object detection performance on LVIS (results from official repo): | Model | Pre-Train Data | MiniVal APr | MiniVal APc | MiniVal APf | MiniVal AP | Val1.0 APr | Val1.0 APc | Val1.0 APf | Val1.0 AP | | ------------------------------------------------------------------------------------------------------------------------------ | --------------------- | ----------- | ----------- | ----------- | ----------- | ---------- | ---------- | ---------- | ----------- | | mmgroundingdinotinyo365v1goldg | O365,GoldG | 28.1 | 30.2 | 42.0 | 35.7(+6.9) | 17.1 | 22.4 | 36.5 | 27.0(+6.9) | | mmgroundingdinotinyo365v1goldggrit | O365,GoldG,GRIT | 26.6 | 32.4 | 41.8 | 36.5(+7.7) | 17.3 | 22.6 | 36.4 | 27.1(+7.0) | | mmgroundingdinotinyo365v1goldgv3det | O365,GoldG,V3Det | 33.0 | 36.0 | 45.9 | 40.5(+11.7) | 21.5 | 25.5 | 40.2 | 30.6(+10.5) | | mmgroundingdinotinyo365v1goldggritv3det | O365,GoldG,GRIT,V3Det | 34.2 | 37.4 | 46.2 | 41.4(+12.6) | 23.6 | 27.6 | 40.5 | 31.9(+11.8) |

MM Grounding DINO model was proposed in An Open and Comprehensive Pipeline for Unified Object Grounding and Detection by Xiangyu Zhao, Yicheng Chen, Shilin Xu, Xiangtai Li, Xinjiang Wang, Yining Li, Haian Huang. MM Grounding DINO improves upon the Grounding DINO by improving the contrastive class head and removing the parameter sharing in the decoder, improving zero-shot detection performance on both COCO (50.6(+2.2) AP) and LVIS (31.9(+11.8) val AP and 41.4(+12.6) minival AP). You can find all the original MM Grounding DINO checkpoints under the MM Grounding DINO collection. You can use the raw model for zero-shot object detection. Here's how to use the model for zero-shot object detection: This model was trained on: - Objects365v1 - GOLD-G - V3Det - COCO 2017 - LVIS v1.0 - COCO 2014 - GRIT - RefCOCO - RefCOCO+ - RefCOCOg - gRefCOCO - Here's a table of models and their object detection performance results on COCO (results from official repo): | Model | Backbone | Pre-Train Data | Style | COCO mAP | | ------------------------------------------------------------------------------------------------------------------------------ | -------- | ------------------------ | --------- | ---------- | | mmgroundingdinotinyo365v1goldg | Swin-T | O365,GoldG | Zero-shot | 50.4(+2.3) | | mmgroundingdinotinyo365v1goldggrit | Swin-T | O365,GoldG,GRIT | Zero-shot | 50.5(+2.1) | | mmgroundingdinotinyo365v1goldgv3det | Swin-T | O365,GoldG,V3Det | Zero-shot | 50.6(+2.2) | | mmgroundingdinotinyo365v1goldggritv3det | Swin-T | O365,GoldG,GRIT,V3Det | Zero-shot | 50.4(+2.0) | | mmgroundingdinobaseo365v1goldgv3det | Swin-B | O365,GoldG,V3Det | Zero-shot | 52.5 | | mmgroundingdinobaseall | Swin-B | O365,ALL | - | 59.5 | | mmgroundingdinolargeo365v2oiv6goldg | Swin-L | O365V2,OpenImageV6,GoldG | Zero-shot | 53.0 | | mmgroundingdinolargeall | Swin-L | O365V2,OpenImageV6,ALL | - | 60.3 | - Here's a table of MM Grounding DINO tiny models and their object detection performance on LVIS (results from official repo): | Model | Pre-Train Data | MiniVal APr | MiniVal APc | MiniVal APf | MiniVal AP | Val1.0 APr | Val1.0 APc | Val1.0 APf | Val1.0 AP | | ------------------------------------------------------------------------------------------------------------------------------ | --------------------- | ----------- | ----------- | ----------- | ----------- | ---------- | ---------- | ---------- | ----------- | | mmgroundingdinotinyo365v1goldg | O365,GoldG | 28.1 | 30.2 | 42.0 | 35.7(+6.9) | 17.1 | 22.4 | 36.5 | 27.0(+6.9) | | mmgroundingdinotinyo365v1goldggrit | O365,GoldG,GRIT | 26.6 | 32.4 | 41.8 | 36.5(+7.7) | 17.3 | 22.6 | 36.4 | 27.1(+7.0) | | mmgroundingdinotinyo365v1goldgv3det | O365,GoldG,V3Det | 33.0 | 36.0 | 45.9 | 40.5(+11.7) | 21.5 | 25.5 | 40.2 | 30.6(+10.5) | | mmgroundingdinotinyo365v1goldggritv3det | O365,GoldG,GRIT,V3Det | 34.2 | 37.4 | 46.2 | 41.4(+12.6) | 23.6 | 27.6 | 40.5 | 31.9(+11.8) |