Cseti

VibeVoice7BHunv2 This is my newest finetuned VibeVoice 7B (Large) model tailored to Hungarian language. I trained LoRA for the LLM module, performed a full-finetune on the Diffusion head modules, and merged each of them into the base model. Thank you for JPGallegoar for that amazing VibeVoice trainer! Inference For inference, you can use - this Comfyui node - Demo codes on VibeVoice Community's repository

Trainig details: I tried a method where I first trained the LoRA on images only and then after a certain number of steps continued training only on videos. - LR: 2e-5 - Optimizer: adamwoptimi - epochs: 15 (image) + 7 (video) - steps: 2550 (image) + 2520 (video) - dataset: 33 images and 35 videos - repeats: 10 - rank: 128 - batch size: 2 - gradient accumulation steps: 1 It seems, the model likes long descriptive prompts. Look at the attached videos for prompt examples. Based on my tests if you use short prompts, the lora effect is weak. For inference I recommend: - ComfyUI native nodes - Kijai's ComfyUI wrapper nodes (If it isn't enough so you don't get the style/character, I recommend adding "animation style" to you prompt. It can help providing the style but in some cases the result will be too cartoony) Important Notes: This LoRA is created as part of a fan project for research purposes only and is not intended for commercial use. It is based on the TV series called Arcane which are protected by copyright. Users utilize the model at their own risk. Users are obligated to comply with copyright laws and applicable regulations. The model has been developed for non-commercial purposes, and it is not my intention to infringe on any copyright. I assume no responsibility for any damages or legal consequences arising from the use of the model. Special thanks to Kijai for the tireless and excelent work done for the community to enable us to use these solutions as soon as possible, as well, as to Comfyanonymous for implementing them so quickly with such a good quality natively into ComfyUI, and also to TDRussel for making LoRA training available to the community so quickly.

Inference NOTE: Use it only on your high noise sampler IMPORTANT: This LoRA is very sensitive to the prompt structure. There are certain phrases that I strongly recommend including in your prompt. Like: - "Sequence of quick scene transitions of" - "The sequence of changes includes the transitions" You can find prompt examples in the attached videos. Trigger words: Banostasis concept, (and the phrases above) HIGH noise LoRA - dataset: 33 videos 480x270 25,33,65,81 frame videos - steps: 15147 - LR: 5e-5 - optimizer: AdamW Optimi - rank: 32 - batch size: 1 - gradient accumulation steps: 1 - mint = 0.875 - maxt = 1

Description: Trainig details: Trained only on videos. - dataset: 87 videos 570x405 81,65,33 frame videos - steps: 15660 - LR: 8e-5 - optimizer: AdamW8bitKahan - rank: 32 - batch size: 1 - gradient accumulation steps: 1 Important Notes: This LoRA is created as part of a fan project for research purposes only and is not intended for commercial use. It is based on the movies called Wallace and Gromit, which are protected by copyright. Users utilize the model at their own risk. Users are obligated to comply with copyright laws and applicable regulations. The model has been developed for non-commercial purposes, and it is not my intention to infringe on any copyright. I assume no responsibility for any damages or legal consequences arising from the use of the model.

VibeVoice7BDiffusion-head-LoRAHungarian-CV17 This is a VibeVoice 7B (Large) model LoRA finetune on a Hungarian audio dataset. For this particular test I used the CommonVoice 17.0 dataset's Hungarian config's train split. To finetune the model I used the following code base. Thank you for JPGallegoar for that amazing VibeVoice trainer! Inference To use the LoRA model you can use my modified fork until the following PR will be merged into the main branch of VibeVoice Community's repository.