Yehor

--- base_model: facebook/wav2vec2-xls-r-300m language: - uk license: "apache-2.0" tags: - automatic-speech-recognition datasets: - mozilla-foundation/common_voice_10_0 metrics: - wer model-index: - name: w2v-xls-r-uk results: - task: name: Automatic Speech Recognition type: automatic-speech-recognition dataset: name: common_voice_10_0 type: common_voice_10_0 config: uk split: test args: uk metrics: - name: WER type: wer value: 20.24 - name: CER type: cer value: 3.64 ---

A lightweight model to do machine translation from English to Ukrainian based on recently published LFM2 model. Use demo to test it. Facts: - Fine-tuned with 40M samples (filtered by quality metric) from ~53.5M for 1.4 epochs - 354M params - Requires 1 GB of RAM to run with bf16 - BLEU on FLORES-200: 27.24 - Tokens per second: 229.93 (bs=1), 1664.40 (bs=10), 8392.48 (bs=64) - License: lfm1.0 Info: - Model name is inherited from name of Sergiy Kulyk who was chargé d'affaires of Ukraine in the United States - Learning Rate: 3e-5 - Learning Rate scheduler type: cosine - Warmup Ratio: 0.05 - Max length: 2048 - Batch Size: 10 - `packed=True` - Sentences <= 1000 chars - Gradient accumulation steps: 4 - Used Flash Attention 2 - Time for epoch: 32 hours - 2 cards of NVIDIA RTX 3090 Ti (24G) - `accelerate` with DeepSpeed, offloading into CPU - Memory usage: 22.212GB-22.458GB - torch 2.7.1 - Serhiy Stetskovych for providing compute to train this model - lang-uk members for their compilation of different MT datasets

A lightweight model to do machine translation from Ukrainian to English based on recently published LFM2 model. Use demo to test it. Facts: - Fine-tuned with 40M samples (filtered by quality metric) from ~53.5M for 1.4 epochs - 354M params - Requires 1 GB of RAM to run with bf16 - BLEU on FLORES-200: 36.27 - Tokens per second: 229.93 (bs=1), 1664.40 (bs=10), 8392.48 (bs=64) - License: lfm1.0 Info: - Model name is inherited from name of Sergiy Kulyk who was chargé d'affaires of Ukraine in the United States - Learning Rate: 3e-5 - Learning Rate scheduler type: cosine - Warmup Ratio: 0.05 - Max length: 2048 - Batch Size: 10 - `packed=True` - Sentences <= 1000 chars - Gradient accumulation steps: 4 - Used Flash Attention 2 - Time for epoch: 32 hours - 2 cards of NVIDIA RTX 3090 Ti (24G) - `accelerate` with DeepSpeed - Memory usage: 22.212GB-22.458GB - torch 2.7.1 - Dmytro Chaplynskyi for providing compute to train this model - lang-uk members for their compilation of different MT datasets

- Discord: https://bit.ly/discord-uds - Speech Recognition: https://t.me/speechrecognitionuk - Speech Synthesis: https://t.me/speechsynthesisuk See other Ukrainian models: https://github.com/egorsmkv/speech-recognition-uk This is the model - https://huggingface.co/Yehor/w2v-bert-uk-v2.1 - where tensors are saved in BF16 format.