TinyStories-33M — ONNX (fp32 + q4) for transformers.js / WebGPU

ONNX export of roneneldan/TinyStories-33M prepared for client-side WebGPU inference via @huggingface/transformers.

Files

Export recipe

Mirrors the pipeline in AlexWortega/PhysicsLLMEngine/inference:

optimum-cli export onnx \
    --model roneneldan/TinyStories-33M \
    --task text-generation-with-past \
    --opset 18 \
    ./out_onnx

Then 4-bit quantization:

from onnxruntime.quantization.matmul_nbits_quantizer import MatMulNBitsQuantizer
quant = MatMulNBitsQuantizer("model.onnx", bits=4, block_size=32, is_symmetric=True)
quant.process()
quant.model.save_model_to_file("model_q4.onnx", use_external_data_format=False)

The q4f16 step from the original recipe was skipped: applying onnxconverter-common.float16.convert_float_to_float16 on top of the post-quantization graph deadlocked (>60 min wall on a 33M model, the MatMulNBits custom op interacted badly with the type rewriter). Plain dtype: "q4" runs cleanly under the WebGPU EP, which is what the demo uses.

Usage (transformers.js)

import { pipeline } from "@huggingface/transformers";
const gen = await pipeline("text-generation", "AlexWortega/tinystories-33m-onnx", {
    device: "webgpu",
    dtype: "q4",
});
const out = await gen("Once upon a time, the little girl", {
    max_new_tokens: 120, temperature: 0.8, top_k: 40, do_sample: true,
});
console.log(out[0].generated_text);

Live demo: AlexWortega/ml-intern-v4-100m-tinystories-demo (Space slug kept from the earlier PyTorch experiment; same URL, now serves this WebGPU build).