talkie-1930-13b-it — GGUF

GGUF port of talkie-lm/talkie-1930-13b-it, the 1930-era vintage language model by Alec Radford et al.

This is the first GGUF release of talkie (v0.x). It was produced by PocketAI; converter source and a forked llama.cpp with LLM_ARCH_TALKIE support are at github.com/PocketAIHub/talkie-gguf.

Important — read before downloading

Stock llama.cpp does not yet know the talkie architecture, and on macOS Metal it requires an environment variable to avoid producing NaN logits at prompt lengths ≥ 9 tokens. You must build PocketAI's llama.cpp fork from source until the upstream PR merges. Both quants below have been compared against the reference PyTorch model: tokenization is byte-perfect, top-1 agrees on 14/15 prompts, generation is coherent in practice. There is non-trivial logit-Δ on chat-template prompts (graph-level drift, same in bf16 and Q8_0 — not quantization noise) that will be tightened up in a future release. See "Validation" below for full numbers.

git clone https://github.com/PocketAIHub/talkie-gguf
cd talkie-gguf/llama.cpp
cmake -B build -DGGML_METAL=ON     # Linux/Windows: omit -DGGML_METAL=ON
cmake --build build -j --target llama-cli

# Run (Metal: env var REQUIRED; CPU/CUDA: optional, no effect):
GGML_METAL_NE11_MM_MIN=1024 \
  ./build/bin/llama-cli -m /path/to/talkie-1930-13b-it-Q8_0.gguf -ngl 99 \
  -p "Among the great inventions of our age,"

Files

Sample output

Prompt: Among the great inventions of our age, the wireless radio has (Q8_0, M1 Pro, --temp 0.7 --seed 42):

made a powerful appeal to the imagination. The fact that it has become possible to talk from continent to continent, from America to Europe, and from ship to shore, has been hailed as little less than miraculous. Yet the marvel of it is small in comparison with the greater marvel that we are able to think from continent to continent, and from generation to generation. The thoughts of great minds in the Old…

Validation

15-prompt logit comparison vs the reference talkie PyTorch model (talkie-lm/talkie-1930-13b-it). Reference run on CPU bf16 (the 26 GB bf16 model does not fit in 24 GB VRAM); GGUF side ran on RTX 3090 (Q8_0, -ngl 99) and on CPU (bf16, too large for a single 3090). Full reports in validation/.

Drift is concentrated on chat-template prompts (<|user|>...<|end|><|assistant|>): non-chat prompts agree within ≤ 0.5 logit units; chat prompts spike to 2-6 on individual logits but the top-1 token still wins on 14/15 chat prompts. Q8_0 and bf16 GGUFs drift the same amount, so the cause is in the graph rather than quantization. Investigation will continue for a future release.

Performance

Apple M1 Pro, 32 GB unified memory, full Metal offload:

NVIDIA RTX 3090, 24 GB VRAM, full CUDA offload (-ngl 99):

About talkie

Talkie-1930 is a vintage-style language model trained on a corpus of 1930-era text. It is a 13B-parameter, 40-layer / 40-head decoder-only transformer with several non-standard architectural choices: scaleless RMSNorm everywhere, post-RoPE Q/K RMSNorm, NeoX-style RoPE with flipped sin signs, SwiGLU MLP, and a per-layer "embed-skip" connection that adds the original embedding (post a pre-block RMSNorm) to the residual stream every layer.

The original release page is talkie-lm/talkie-1930-13b-it. The reference code lives at github.com/talkie-lm/talkie.

License

Apache 2.0 — same as the original. See NOTICE in the github fork for attribution and a list of PocketAI's modifications.

Why the env var is needed (technical detail)

On Metal at batch size ≥ 9, llama.cpp's kernel_mul_mm_q*_f32 stores both operands of the matrix multiply in shared memory as simdgroup_half8x8 (fp16 tiles). Talkie's scaleless RMSNorm produces F32 SwiGLU activations that occasionally exceed fp16 max (±65 504), which overflow to ±inf during the F32→half cast and propagate to NaN logits. The env var raises the matrix-vector → matrix-matrix kernel-selection threshold so all batched matmuls stay on mul_mv_ext / mul_mv, which keep operands in F32 in shared memory. CPU and CUDA backends do not exhibit this issue.

The fix has no effect on other models — unset, behavior matches stock llama.cpp exactly. Discussion and the upstream PR will be linked here once filed.