vitouphy

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the openslr dataset. It achieves the following results on the evaluation set: - Loss: 0.3281 - Wer: 0.3462 Evaluation results on OpenSLR "test" (self-split 10%) (Running ./eval.py): - WER: 0.3216977389924633 - CER: 0.08653361193169537 Evaluation results with language model on OpenSLR "test" (self-split 10%) (Running ./eval.py): - WER: 0.257040856802856 - CER: 0.07025001801282513 Installation Install the following libraries on top of HuggingFace Transformers for the supports of language model. Approach 1: Using HuggingFace's pipeline, this will cover everything end-to-end from raw audio input to text output. The data used for this model is only around 4 hours of recordings. - We split into 80/10/10. Hence, the training hour is 3.2 hours, which is very very small. - Yet, its performance is not too bad. Quite interesting for such small dataset, actually. You can try it out. - Its limitation is: - Rare characters, e.g. ឬស្សី ឪឡឹក - Speech needs to be clear and articulate. - More data to cover more vocabulary and character may help improve this system. The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 8 - evalbatchsize: 8 - seed: 42 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 32 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lrschedulertype: linear - lrschedulerwarmupsteps: 1000 - numepochs: 100 - mixedprecisiontraining: Native AMP | Training Loss | Epoch | Step | Validation Loss | Wer | |:-------------:|:-----:|:----:|:---------------:|:------:| | 5.0795 | 5.47 | 400 | 4.4121 | 1.0 | | 3.5658 | 10.95 | 800 | 3.5203 | 1.0 | | 3.3689 | 16.43 | 1200 | 2.8984 | 0.9996 | | 2.01 | 21.91 | 1600 | 1.0041 | 0.7288 | | 1.6783 | 27.39 | 2000 | 0.6941 | 0.5989 | | 1.527 | 32.87 | 2400 | 0.5599 | 0.5282 | | 1.4278 | 38.35 | 2800 | 0.4827 | 0.4806 | | 1.3458 | 43.83 | 3200 | 0.4429 | 0.4532 | | 1.2893 | 49.31 | 3600 | 0.4156 | 0.4330 | | 1.2441 | 54.79 | 4000 | 0.4020 | 0.4040 | | 1.188 | 60.27 | 4400 | 0.3777 | 0.3866 | | 1.1628 | 65.75 | 4800 | 0.3607 | 0.3858 | | 1.1324 | 71.23 | 5200 | 0.3534 | 0.3604 | | 1.0969 | 76.71 | 5600 | 0.3428 | 0.3624 | | 1.0897 | 82.19 | 6000 | 0.3387 | 0.3567 | | 1.0625 | 87.66 | 6400 | 0.3339 | 0.3499 | | 1.0601 | 93.15 | 6800 | 0.3288 | 0.3446 | | 1.0474 | 98.62 | 7200 | 0.3281 | 0.3462 | - Transformers 4.17.0.dev0 - Pytorch 1.10.2+cu102 - Datasets 1.18.2.dev0 - Tokenizers 0.11.0

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the openslr dataset. It achieves the following results on the evaluation set: - Loss: 0.4239 - Wer: 0.4221 Evaluation results on OpenSLR "test" (self-split 10%) (Running ./eval.py): - WER: 0.4490281634272114 - CER: 0.12198285179047481 Evaluation results on OpenSLR "test" with LM ngram (self-split 10%) (Running ./eval.py): - WER: 0.32130107100357 - CER: 0.09345053678218891 Note - Since this dataset is small (4 hours of voice recording), we decided not to train that for too long to avoid overfitting and under-generalization. - This model performs worse than its 300M-variant. Probably, we don't explore the hyper-parameter enough? Installation Install the following libraries on top of HuggingFace Transformers for the supports of language model. Approach 1: Using HuggingFace's pipeline, this will cover everything end-to-end from raw audio input to text output. The data used for this model is only around 4 hours of recordings. - We split into 80/10/10. Hence, the training hour is 3.2 hours, which is very very small. - Yet, its performance is not too bad. Quite interesting for such small dataset, actually. You can try it out. - Its limitation is: - Rare characters, e.g. ឬស្សី ឪឡឹក - Speech needs to be clear and articulate. - More data to cover more vocabulary and character may help improve this system. The following hyperparameters were used during training: - learningrate: 1e-05 - trainbatchsize: 8 - evalbatchsize: 8 - seed: 42 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 32 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lrschedulertype: linear - lrschedulerwarmupsteps: 2000 - numepochs: 75 - mixedprecisiontraining: Native AMP | Training Loss | Epoch | Step | Validation Loss | Wer | |:-------------:|:-----:|:----:|:---------------:|:------:| | 3.5671 | 5.47 | 400 | 12.0218 | 1.0 | | 3.5159 | 10.95 | 800 | 10.6337 | 1.0 | | 2.4543 | 16.43 | 1200 | 1.8256 | 0.9839 | | 1.9437 | 21.91 | 1600 | 1.1237 | 0.9173 | | 1.696 | 27.39 | 2000 | 0.8246 | 0.7700 | | 1.5342 | 32.87 | 2400 | 0.6433 | 0.6594 | | 1.4509 | 38.35 | 2800 | 0.5500 | 0.5787 | | 1.3478 | 43.83 | 3200 | 0.5070 | 0.4907 | | 1.3096 | 49.31 | 3600 | 0.4692 | 0.4726 | | 1.2532 | 54.79 | 4000 | 0.4448 | 0.4479 | | 1.2291 | 60.27 | 4400 | 0.4374 | 0.4366 | | 1.196 | 65.75 | 4800 | 0.4314 | 0.4310 | | 1.1862 | 71.23 | 5200 | 0.4239 | 0.4221 | - Transformers 4.17.0.dev0 - Pytorch 1.10.2+cu102 - Datasets 1.18.2.dev0 - Tokenizers 0.11.0