seastar105

적지 않은 분들이 다운로드하셔서 사용하는 걸로 보입니다. https://huggingface.co/seastar105/whisper-medium-komixv2 에 더 좋은 정확도를 기대 가능한 파인튜닝 모델이 있으므로 이쪽을 추천합니다. This model is a fine-tuned version of openai/whisper-medium on the Zeroth Korean dataset. It achieves the following results on the evaluation set: - Loss: 0.0727 - Wer: 3.6440 - Cer: 1.4840 The following hyperparameters were used during training: - learningrate: 5e-06 - trainbatchsize: 8 - evalbatchsize: 8 - seed: 42 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 16 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lrschedulertype: linear - lrschedulerwarmupsteps: 500 - trainingsteps: 5000 - mixedprecisiontraining: Native AMP | Training Loss | Epoch | Step | Validation Loss | Wer | Cer | |:-------------:|:-----:|:----:|:---------------:|:------:|:------:| | 0.0873 | 0.72 | 1000 | 0.1086 | 7.7549 | 2.5597 | | 0.0258 | 1.44 | 2000 | 0.0805 | 4.5475 | 1.7588 | | 0.0091 | 2.16 | 3000 | 0.0719 | 3.7946 | 1.5664 | | 0.0086 | 2.88 | 4000 | 0.0704 | 3.5537 | 1.5232 | | 0.0019 | 3.59 | 5000 | 0.0727 | 3.6440 | 1.4840 | - Transformers 4.26.0.dev0 - Pytorch 1.13.0a0+d0d6b1f - Datasets 2.7.1 - Tokenizers 0.13.2

This model is fine-tuned from microsoft/Phi-4-multimodal-instruct on kresnik/zerothkorean dataset only 1 epoch. script for fine-tuning is here, adapted from phi-4 repository example model is trained only 174 steps on zeroth train set, and main purpose is to check if only korean ASR training can expand to other speech tasks(e.g. speech-to-text-translation) ASR on zeroth-test set and Speech translation on fleurs ko en speech translation result. script is here, and used 1 A40. | Model | zeroth-test | fleurs-ko2en | fleurs-ko2en-cot | fleurs-en2ko | fleurs-en2ko-cot | |----------|------------|--------------|------------------|--------------|------------------| | original | 195.92 | 5.62 | 2.45 | 6.87 | 4.35 | | finetune (this model) | 7.02 | 7.07 | 9.19 | 13.08 | 9.35 |