https://huggingface.co/CNWPlayer/VegaLM1-42M-Base further trained on another roughly 2.8B tokens of stack-v3-train. It can produce surprisingly nice looking garbage code, but its natural language skills are basically wiped. Dare I say, it's SOTA for <90M coding models?
