Loom: A Scalable Computer Architecture for Looped Transformers

Paper | GitHub | Live Demos

Loom is a general-purpose computer implemented as a looped transformer with analytically derived weights. Programs are written in C, compiled to a 21-opcode ISA, and executed as iterated matrix multiplications through 8 fixed-weight transformer layers.

Model Description

Loom is not a trained model. Every weight is derived analytically from the ISA specification. The transformer implements a programmable computer: each forward pass executes one compiled instruction, and the model is applied in a loop until the program halts.

The architecture supports 21 opcodes (arithmetic, logic, shifts, comparisons, branches, indirect memory access, conditional moves, and multiply-accumulate) in 8 transformer layers, down from the 10 layers required for the single-instruction baseline of Giannou et al.

Key Design Choices

• Argmax attention replaces softmax for numerically exact execution over arbitrary step counts.
• Opcode-as-operand-routing maps all 21 operations to operand preparation for a shared subtract core, requiring only one arithmetic layer.
• 6-threshold direct subtraction computes a-b in one layer (replacing the classical 3-layer approach).
• STORE opcode enables indirect memory writes, reducing the Sudoku solver from 1,085 to 284 instructions.

Configurations

ONNX Models

The repository includes pre-exported ONNX models with argmax attention expressed as GPU-native operations (ReduceMax, comparison, division). No TopK or OneHot operators. Full WebGPU acceleration.

• argmax_146x512.onnx (8.3 MB): compact config
• argmax_155x1024.onnx (14.8 MB): standard config
• argmax_164x2048.onnx (28.0 MB): large config

Download all three:

huggingface-cli download mehmetkeremturkcan/Loom --local-dir Loom

Input/Output

• Input: state tensor of shape [d_model, n], dtype float32
• Output: new_state tensor of shape [d_model, n], dtype float32

Each call executes one ISA instruction. Loop until PC (program counter) reaches 0.

Usage

Quick Start (Python)

from loom_v1 import LoomConfig, LoomComputer, init_state, read_memory, get_pc, OP_INC, OP_HALT
from subleq import signed_from_bipolar
import torch

cfg = LoomConfig(s=32, m=8, n=64, N=8)
comp = LoomComputer(cfg)
X = init_state(cfg, [5,0,0,0,0,0,0,0], [(OP_INC, cfg.s, 0), (OP_HALT, 0, 0)])

with torch.no_grad():
    while get_pc(X, cfg) != 0:
        X = comp.step(X)

print('mem[0] =', read_memory(X, cfg)[0])  # 6

Full C Compilation

from loom_v1 import LoomConfig, LoomComputer, init_state, read_memory, get_pc
from c_compiler import compile_c
import torch

source = """
int main() {
    int a; int b; int t; int i;
    a = 0; b = 1; i = 0;
    while (i < 10) { t = a + b; a = b; b = t; i = i + 1; }
    return a;
}
"""

cfg, mem, cmds, meta = compile_c(source, s=32, m=160, n=512, N=8)
comp = LoomComputer(cfg)
X = init_state(cfg, mem, cmds)

with torch.no_grad():
    while get_pc(X, cfg) != 0:
        X = comp.step(X)

from subleq import signed_from_bipolar
a = signed_from_bipolar(X[cfg.idx_memory:cfg.idx_memory+cfg.N, cfg.s + meta['variables']['a']])
print(f"fib(10) = {a}")  # 55

Validation

• 42 opcode unit tests (all pass)
• 19 SWAP integration tests (all pass)
• 50 compiled C program tests including Fibonacci, GCD, sorting, LOAD/STORE round-trips (all pass)
• FPGA hardware verification on Xilinx Alveo U200 (INC test pass)

Browser Demos

Interactive demos run entirely client-side via ONNX Runtime Web:

• Sorting with real-time architecture visualization (3D layer activations)
• C debugger with source highlighting and variable watch
• Playable Snake game (84 transformer steps per tick)
• 9x9 Sudoku solver
• DOOM raycasting

Limitations

• 8-bit signed integers (-128 to 127) by default
• One instruction per forward pass (no pipelining)
• No multiplication/division in hardware (software emulation via MULACC)
• FIND requires unique values in the search array
• LOAD/STORE require valid in-range pointers

Citation

@misc{turkcan2026loomscalableanalyticalneural,
      title={Loom: A Scalable Analytical Neural Computer Architecture}, 
      author={Mehmet Kerem Turkcan},
      year={2026},
      eprint={2604.08816},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2604.08816}, 
}