N7-T5-small-EN-MNI
It achieves the following results on the evaluation set:
- Loss: 3.3832
- Bleu: 2.1755
- Gen Len: 127.0
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 2000
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|---|---|---|---|---|---|
| 5.7624 | 1.0 | 10381 | 5.4722 | 0.6341 | 126.9983 |
| 5.0926 | 2.0 | 20762 | 4.7662 | 1.0765 | 126.9999 |
| 4.6415 | 3.0 | 31143 | 4.2904 | 1.4149 | 127.0 |
| 4.2391 | 4.0 | 41524 | 3.9571 | 1.663 | 127.0 |
| 4.0196 | 5.0 | 51905 | 3.7264 | 1.8588 | 127.0 |
| 3.8029 | 6.0 | 62286 | 3.5647 | 2.0018 | 127.0 |
| 3.7119 | 7.0 | 72667 | 3.4628 | 2.0933 | 127.0 |
| 3.664 | 8.0 | 83048 | 3.4082 | 2.1472 | 127.0 |
| 3.7063 | 9.0 | 93429 | 3.3857 | 2.1719 | 127.0 |
| 3.6087 | 10.0 | 103810 | 3.3832 | 2.1755 | 127.0 |
EXAMPLE USAGE
from transformers import AutoTokenizer
from transformers import AutoModelForSeq2SeqLM
model = 'DayanandaThokchom/N7-T5-small-EN-MNI'
model = AutoModelForSeq2SeqLM.from_pretrained(model)
tokenizer = AutoTokenizer.from_pretrained(model)
text = "translate English to meitei mayek: what is the most important things for you"
inputs = tokenizer(text, return_tensors="pt").input_ids
# Generate text based on the input
outputs = model.generate(
inputs, # The tokenized input text
max_new_tokens=50, # The maximum number of new tokens to generate
do_sample=True, # Whether to use sampling for generation
top_k=10, # In sampling, consider only the top_k most likely tokens
top_p=0.50 # In sampling, consider the smallest set of tokens whose cumulative probability exceeds top_p
)
decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
cleaned_output = decoded_output.split('.')[0] if '.' in decoded_output else decoded_output
print(f"result:- {cleaned_output}")
Framework versions
- Transformers 4.55.0
- Pytorch 2.7.1+cu128
- Datasets 4.0.0
- Tokenizers 0.21.4