whisper-large-v2-translate-zh-v0.1-lt-ct2
This model is a fine-tuned version of openai/whisper-large-v2.
Model description
3500小时 (日语音频,中文字幕) 数据微调, 翻译模式直出中文
Usage
task='translate', language='ja'
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 4000
- dropout: 0.1
- mask_time_prob: 0.05
- mask_feature_prob: 0.2
- condition_on_previous_text_rate: 0.5
Training results
| Training Loss | Epoch | Step | Validation Loss | Cer | Wer |
|---|---|---|---|---|---|
| 1.743 | 0.0740 | 1000 | 1.5631 | 0.8223 | 1.4517 |
| 1.6014 | 0.1479 | 2000 | 1.4808 | 0.6775 | 1.0950 |
| 1.5549 | 0.2219 | 3000 | 1.4381 | 0.6756 | 1.1158 |
| 1.5283 | 0.2958 | 4000 | 1.4174 | 0.6992 | 1.1137 |
| 1.474 | 0.3698 | 5000 | 1.3849 | 0.6570 | 1.1369 |
| 1.4193 | 0.4437 | 6000 | 1.3657 | 0.6544 | 1.1339 |
| 1.4148 | 0.5177 | 7000 | 1.3477 | 0.6386 | 1.1647 |
| 1.3754 | 0.5916 | 8000 | 1.3392 | 0.6228 | 1.0461 |
| 1.3441 | 0.6656 | 9000 | 1.3362 | 0.6196 | 1.0609 |
| 1.3545 | 0.7395 | 10000 | 1.3176 | 0.6354 | 1.2138 |
| 1.3498 | 0.8135 | 11000 | 1.3236 | 0.6631 | 1.2232 |
| 1.31 | 0.8874 | 12000 | 1.3020 | 0.6199 | 1.0018 |
| 1.3213 | 0.9614 | 13000 | 1.2966 | 0.5922 | 1.0021 |
| 1.2375 | 1.0353 | 14000 | 1.2900 | 0.6097 | 1.0639 |
| 1.2334 | 1.1093 | 15000 | 1.2963 | 0.6150 | 1.0920 |
| 1.2277 | 1.1832 | 16000 | 1.2888 | 0.6077 | 1.0929 |
| 1.2087 | 1.2572 | 17000 | 1.2779 | 0.5954 | 1.0012 |
| 1.2131 | 1.3311 | 18000 | 1.2722 | 0.5776 | 1.0075 |
| 1.2012 | 1.4051 | 19000 | 1.2716 | 0.5726 | 1.0211 |
| 1.1912 | 1.4790 | 20000 | 1.2707 | 0.6007 | 1.1538 |
| 1.2127 | 1.5530 | 21000 | 1.2749 | 0.6086 | 1.0742 |
| 1.1789 | 1.6269 | 22000 | 1.2797 | 0.5765 | 1.0072 |
| 1.1527 | 1.7009 | 23000 | 1.2761 | 0.5855 | 1.0588 |
| 1.1693 | 1.7748 | 24000 | 1.2701 | 0.5635 | 0.9928 |
| 1.1709 | 1.8488 | 25000 | 1.2662 | 0.5980 | 1.0697 |
| 1.1637 | 1.9227 | 26000 | 1.2749 | 0.5872 | 1.0392 |
| 1.1562 | 1.9967 | 27000 | 1.2587 | 0.5651 | 1.0121 |
| 1.0929 | 2.0706 | 28000 | 1.2668 | 0.5857 | 1.0139 |
| 1.1232 | 2.1446 | 29000 | 1.2710 | 0.5742 | 0.9997 |
| 1.1045 | 2.2185 | 30000 | 1.2656 | 0.5643 | 0.9897 |
| 1.0841 | 2.2925 | 31000 | 1.2695 | 0.5835 | 1.0181 |
| 1.0868 | 2.3664 | 32000 | 1.2707 | 0.5673 | 0.9964 |
| 1.0938 | 2.4404 | 33000 | 1.2644 | 0.5712 | 0.9928 |
| 1.0938 | 2.5143 | 34000 | 1.2662 | 0.5750 | 1.0109 |
| 1.0848 | 2.5883 | 35000 | 1.2677 | 0.5841 | 1.0832 |
| 1.0914 | 2.6622 | 36000 | 1.2638 | 0.5801 | 1.0299 |
| 1.0688 | 2.7362 | 37000 | 1.2587 | 0.5694 | 1.0072 |
| 1.0856 | 2.8101 | 38000 | 1.2581 | 0.5646 | 1.0057 |
| 1.1037 | 2.8841 | 39000 | 1.2557 | 0.5771 | 1.0262 |
| 1.0652 | 2.9580 | 40000 | 1.2566 | 0.5634 | 0.9979 |
Framework versions
- Transformers 4.41.1
- Pytorch 2.3.1+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1