Qwen3-0.6B weight_only_wi4_afp32: GPU crash reproduction

This repo holds the files that reproduce LiteRT-LM issue #2716. The .litertlm bundle is a test case and is not meant for use.

• qwen3-0.6b_weight_only_wi4_afp32.litertlm (315,697,136 B, sha256 4cc6201f38aa6655b48240a2318bd5b0254dc1077be8d951fab26d7256e068ba): Qwen/Qwen3-0.6B, exported with litert-torch 0.9.3 (ai-edge-quantizer 0.8.0) by this command and re-packed on 2026-10-14 (chat template only, see below):

  python -m litert_torch.generative.export_hf Qwen/Qwen3-0.6B <out dir> --quantization_recipe=weight_only_wi4_afp32 --prefill_lengths=128 --cache_length=1024
  

  The graph is the explicit-dequantize form. In prefill_128 (1,493 ops), 192 DEQUANTIZE ops take int4 inputs, and the EMBEDDING_LOOKUP table is the float32 output of one of them. Since the 2026-10-14 re-pack the chat template renders a system message with content parts (the earlier template failed with tried to use + operator on unsupported types string and sequence), and on litert-lm 0.18.0, which hands every message to the template as content parts, it renders each turn's text (the earlier template rendered every turn empty there, so a CPU run answered an empty prompt). Only the LlmMetadata section changed; the tokenizer and the graph are byte-identical to the export, so the GPU crash is unchanged. The table below was measured with the exported file, which stays at commit 8897b5ef; with the re-packed file, the CPU run (exit 0) and the GPU crash (exit 245) were re-checked on litert-lm 0.18.0 (Python API, 2026-10-14).
• crash-litert-lm-0.17.1.ips, crash-litert-lm-0.18.0.ips: macOS crash reports of litert-lm run <bundle> --backend gpu on litert-lm 0.17.1 and 0.18.0. Both show EXC_BAD_ACCESS / SIGSEGV, KERN_INVALID_ADDRESS at 0x0000000000000000, with frame 0 _platform_memmove+52 and frame 1 litert::ml_drift::(anonymous namespace)::EmbeddingLookupOperationParser::Parse(…)+1888. Home paths in both reports are replaced by /Users/USER.

Tested on macOS 27.0, Apple M4 Max:

Issue: google-ai-edge/LiteRT-LM#2716 · LiteRT: google-ai-edge/litert

Known runtime issues

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders this file's chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.