tiny-random-ouro
Tiny random Ouro model for optimum-intel OpenVINO CI.
Architecture: OuroForCausalLM (Universal Transformer). Generated with random weights; the full
vocab_size (49152) and the real tokenizer are kept so the tokenizer round-trips, while all other
dimensions are shrunk. total_ut_steps=4 is preserved to exercise the Universal-Transformer loop.
| field | value |
|---|---|
| hidden_size | 32 |
| intermediate_size | 64 |
| num_hidden_layers | 2 |
| num_attention_heads | 4 |
| num_key_value_heads | 2 |
| head_dim | 8 |
| total_ut_steps | 4 |
| vocab_size | 49152 |
Reproduce with the script in the optimum-intel PR description.