GGUF version:
Q8 tested in the latest CPU llama.cpp build. Disable cuda, vulkan, rocm etc if not CPU version.
I strongly reccomend you use the .bin version and the latest tinylm inference engine. Note: the GGUF version is compatible with llama.cpp but llama.cpp currently does not fully support the chat template, and therefore will produce excessive hallucinations. For optimal usage, please use the .bin version with the latest tinylm.
LS-63M-A16M
| task | 63M / 16M act | 92M / 22M act | 220M / 25M act | chance |
|---|---|---|---|---|
| arc_easy | 31.2% | 35.0% | 35.8% | 25.1% |
| hellaswag | 27.3% | 28.5% | 31.8% | 25.0% |
| piqa | 56.5% | 60.0% | 60.8% | 50.0% |
| lambada | 13.2% | 18.5% | 19.2% | 0% |
| mmlu | 24.8% | 23.8% | 24.5% | 25.0% |
Miniature mixture of experts model with top-1 routing. Trained entirely on a 1660 super. This checkpoint marks the first epoch of training complete, ~1B tokens over 30 GPU hours.
Total parameters: 63M Active parameters: 16M context length: 8192
Training end evals:
| val loss | 1.4286 |
|---|---|
| perplexity | 4.17 |
Chat:
you> hi
bot> Hello! How can I assist you today?
[13 tok, 117.1 tok/s, ctx 23/16384]
you> what is the capital of france?
bot> Juan Van Gogh
[12 tok, 129.0 tok/s, ctx 52/16384]
Code:
you> write a python function that reverses a string
bot> Here is a simple Python function that reverses a string:
```python
def reverse_string(s):
return s[::-1]
```
In this function, we use the `re.split()` function to split the string at the commas and create a list of words. Then we use `re.split()` to split the string on the `^`, and finally, we use `str.split()` to split the list of words.
[98 tok, 63.9 tok/s, ctx 116/16384]
To try it yourself:
Download tinylm.exe and LS-63M-A16M-q8.bin (placed in \models), run command tinylm chat LS-63M-A16M-q8 2048
Note: the bundled tinylm.exe is likely outdated. For the latest version, check here for the source code of tinylm and windows prebuilts.