KwaiVGI

📃 [Paper] • 🌐 [Project Page] • [Github] • 🤗 [VideoGen-RewardBench] • 🏆 [ Leaderboard] Introduction Welcome to VideoReward, a VLM-based reward model introduced in our paper Improving Video Generation with Human Feedback. VideoReward is a multi-dimensional reward model that evaluates generated videos on three critical aspects: Visual Quality (VQ): The clarity, aesthetics, and single-frame reasonableness. Motion Quality (MQ): The dynamic stability, dynamic reasonableness, naturalness, and dynamic degress. Text Alignment (TA): The relevance between the generated video and the text prompt. This versatile reward model can be used for data filtering, guidance, reject sampling, DPO, and other RL methods. If you find this project useful, please consider citing: ```bibtex @article{liu2025improving, title={Improving Video Generation with Human Feedback}, author={Jie Liu and Gongye Liu and Jiajun Liang and Ziyang Yuan and Xiaokun Liu and Mingwu Zheng and Xiele Wu and Qiulin Wang and Wenyu Qin and Menghan Xia and Xintao Wang and Xiaohong Liu and Fei Yang and Pengfei Wan and Di Zhang and Kun Gai and Yujiu Yang and Wanli Ouyang}, journal={arXiv preprint arXiv:2501.13918}, year={2025} }

It synthesizes realistic robotic manipulation video given an initial frame, a prompt, a user-defined object mask, and a collaborative trajectory describing the motion of both robotic arm and manipulated object in decomposed interaction phases. It supports diverse manipulation skills and can generalize to in-the-wild scenarios. This is the implementation based on CogVideoX-5B. Please refer to our github for details on usage.