mini-chennus-2

This model is a fine-tuned version of EleutherAI/pythia-14m-deduped on an unknown dataset. It achieves the following results on the evaluation set:

Model description

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
num_epochs: 6

Epoch	Step	Validation Loss
0.1616	200	nan
0.3231	400	nan
0.4847	600	nan
0.6462	800	nan
0.8078	1000	nan
0.9693	1200	nan
1.1309	1400	nan
1.2924	1600	nan
1.4540	1800	nan
1.6155	2000	nan
1.7771	2200	nan
1.9386	2400	nan
2.1002	2600	nan
2.2617	2800	nan
2.4233	3000	nan
2.5848	3200	nan
2.7464	3400	nan
2.9079	3600	nan
3.0695	3800	nan
3.2310	4000	nan
3.3926	4200	nan
3.5541	4400	nan
3.7157	4600	nan
3.8772	4800	nan
4.0388	5000	nan
4.2003	5200	nan
4.3619	5400	nan
4.5234	5600	nan
4.6850	5800	nan
4.8465	6000	nan
5.0081	6200	nan
5.1696	6400	nan
5.3312	6600	nan
5.4927	6800	nan
5.6543	7000	nan
5.8158	7200	nan
5.9774	7400	nan