Qwen3-ASR-1.7B-RKLLM

pip install numpy scipy soundfile tqdm transformers ztu-somemodelruntime-ez-rknn-async

python run_qwen3_asr_e2e.py --audio-path ./long_test.wav

> python run_qwen3_asr_e2e.py --audio-path ./long_test.wav
I rkllm: rkllm-runtime version: 1.2.3, rknpu driver version: 0.9.8, platform: RK3588
I rkllm: loading rkllm model from rknn/language_model.rkllm
I rkllm: rkllm-toolkit version: 1.2.3, max_context_limit: 4096, npu_core_num: 3, target_platform: RK3588, model_dtype: FP16
I rkllm: Enabled cpus: [4, 5, 6, 7]
I rkllm: Enabled cpus num: 4
I rkllm: reset chat template:
I rkllm: system_prompt: <|im_start|>system\n<|im_end|>\n
I rkllm: prompt_prefix: <|im_start|>user\n
I rkllm: prompt_postfix: <|im_end|>\n<|im_start|>assistant\n
W rkllm: Calling rkllm_set_chat_template will disable the internal automatic chat template parsing, including enable_thinking. Make sure your custom prompt is complete and valid.
input_feature_len: 4031
audio_features: (524, 2048)
time_mel_sec: 2.532
time_rkllm_init_sec: 4.404
time_load_total_sec: 8.092
time_audio_encoder_sec: 1.391
language Chinese<asr_text>大家好呀！今天给大家分享的是在线一键语音生成网站的合集，能够更加方便大家选择自己想要生成的角色。进入网站可以看到所有的生成模型都在这里，选择你想要生成的角色，点击进入就来到了生成的页面，在文本框内输入你想要生成的内容，然后点击生成就好了。另外呢，因为每次的生成结果都会有一些不一样的地方，如果您觉得第一次的生成效果不好的话，可以尝试重新生成，也可以稍微调节一下相关的数值再生成试试。使用时一定要遵守法律法规，不可以损害刷人的形象哦！(finish)
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Model init time (ms)  3747.00                                                                    
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Stage         Total Time (ms)  Tokens    Time per Token (ms)      Tokens per Second      
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Prefill       4193.03          539       7.78                     128.55                 
I rkllm:  Generate      15643.47         118       132.57                   7.54                   
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Peak Memory Usage (GB)
I rkllm:  5.03        
I rkllm: --------------------------------------------------------------------------------------
time_generate_sec: 19.872
time_infer_total_sec: 23.673
time_total_sec: 31.765

> python run_qwen3_asr_e2e.py --audio-path ./long_test.wav
I rkllm: rkllm-runtime version: 1.2.3, rknpu driver version: 0.9.8, platform: RK3588
I rkllm: loading rkllm model from rknn/language_model.rkllm
I rkllm: rkllm-toolkit version: 1.2.3, max_context_limit: 4096, npu_core_num: 3, target_platform: RK3588, model_dtype: FP16
I rkllm: Enabled cpus: [4, 5, 6, 7]
I rkllm: Enabled cpus num: 4
I rkllm: reset chat template:
I rkllm: system_prompt: <|im_start|>system\n<|im_end|>\n
I rkllm: prompt_prefix: <|im_start|>user\n
I rkllm: prompt_postfix: <|im_end|>\n<|im_start|>assistant\n
W rkllm: Calling rkllm_set_chat_template will disable the internal automatic chat template parsing, including enable_thinking. Make sure your custom prompt is complete and valid.
input_feature_len: 4031
audio_features: (524, 2048)
time_mel_sec: 2.532
time_rkllm_init_sec: 4.404
time_load_total_sec: 8.092
time_audio_encoder_sec: 1.391
language Chinese<asr_text>大家好呀！今天给大家分享的是在线一键语音生成网站的合集，能够更加方便大家选择自己想要生成的角色。进入网站可以看到所有的生成模型都在这里，选择你想要生成的角色，点击进入就来到了生成的页面，在文本框内输入你想要生成的内容，然后点击生成就好了。另外呢，因为每次的生成结果都会有一些不一样的地方，如果您觉得第一次的生成效果不好的话，可以尝试重新生成，也可以稍微调节一下相关的数值再生成试试。使用时一定要遵守法律法规，不可以损害刷人的形象哦！(finish)
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Model init time (ms)  3747.00                                                                    
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Stage         Total Time (ms)  Tokens    Time per Token (ms)      Tokens per Second      
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Prefill       4193.03          539       7.78                     128.55                 
I rkllm:  Generate      15643.47         118       132.57                   7.54                   
I rkllm: --------------------------------------------------------------------------------------
I rkllm:  Peak Memory Usage (GB)
I rkllm:  5.03        
I rkllm: --------------------------------------------------------------------------------------
time_generate_sec: 19.872
time_infer_total_sec: 23.673
time_total_sec: 31.765