Gemma-4-E4B meeting agent (zh-TW): LiteRT mobile

Gemma-4-E4B fine-tuned to read a live zh-TW meeting transcript on a phone, window by window, and write short cited notes: decisions, actions, proposals, open issues and key figures. The same model turns the notes into a prose summary and a title. It is the E4B sibling of Luigi/gemma-4-E2B-meeting-agent-zh mobile-v1: same prompts, protocol and note format, fewer factual errors, about 3× slower.

Latest: mobile-v2/ (2026-10-17), see the mobile-v2 section. The files at the root are mobile-v1.

Code, agent protocol and evaluation: https://github.com/vieenrose/meeting-summarizer. Integration: integration note.

mobile-v2 (2026-10-17): fine-tuned further, same engine and files

mobile-v2/ holds the E4B mobile-v2 reader. It continues mobile-v1 with two training stages:

1. SFT on reading rows plus dense-list conversion rows, including a third domain: Taiwanese financial TV programs;
2. one round of on-policy distillation. The student reads training meetings with the deployed harness. Gemma-4-31B then gives its top-20 next-token distribution on the student's own rollouts, and the student is trained on that distribution.

The export is the same as mobile-v1: GPTQ onto Google's integer grid, written into Google's mobile graph, then the attention is fused for the forked engine. The embedders and the tokenizer are byte-identical to mobile-v1. Only the main graph and the system prompt change.

The system prompt changes in one line. 每段最多 5 則 becomes 每段最多 8 則: the model was trained and evaluated with up to 8 notes per window. Use mobile-v2/system_prompt.txt with the mobile-v2 weights, and the mobile-v1 prompt with the mobile-v1 weights. Everything else is unchanged: the protocol, the 4k context, about 1,500 transcript tokens per window, the restart journal of 1,200 tokens, read_max 400, and the sampler.

Results against mobile-v1. Same engine (forked, CPU, 4k) and same judge (Gemma-4-31B), on 43 held-out meetings in three domains: 10 financial TV programs, 13 Legislative Yuan committee meetings and 20 AliMeeting business meetings. None of them was used in training.

What this means for E4B. The share of contradicted notes drops by 1.5 points over the 43 meetings (95 % CI −2.9 to −0.2): the improvement is significant, and it is largest on the financial programs. v2 writes more notes, so the count per meeting rises slightly (+1.2). Coverage rises on the financial programs (+0.11) and is flat on committee meetings, but falls on AliMeeting (−0.05, significant). Part of the coverage gain comes from longer summaries. E4B mobile-v2 is the recommended E4B reader, except for apps that mostly summarize business meetings like AliMeeting.

How it was built

• Base graph. Google's Gemma-4-E4B mobile graph (litert-community/gemma-4-E4B-it-litert-lm): int4/int2 weights, an int8 KV cache, and int8 static-range activations.
• Training. A LoRA trained on Google's mobile weights, with Google's activation ranges in the loop.
• Back onto Google's grid. The merged weights are rounded onto Google's own integer grid with GPTQ, using Google's scales. The integers are then written into Google's file, bit for bit.
• Two settings change. GPU activations are fp32, because fp16 corrupts long prompts on the Mali-G68. The prefill_1024 signature is disabled, because its buffers dominate peak memory.

Results

Quality. Measured on 38 held-out sessions of Legislative Yuan committee meetings, transcribed by on-device ASR. The judge is Gemma-4-31B, which checks each cited statement against the transcript. The runs went through LiteRT-LM with an 8k context.

Through the forked engine (CPU, 4k) on 13 of these sessions, E4B gives the same figures: 11 % contradicted, coverage 0.93, 決議事項 75 %, 待辦 76 %. An 8k context adds only decision recall (73 % to 80 %) and costs 10 to 20 % of the speed, so 4k is the recommended setting (E4B peak RSS 2.65 GB at 4k, 2.82 GB at 8k).

Speed and memory. Measured on an OPPO Reno7 (Dimensity 900, 8 GB), with a 4k context and a 2,881-token reading prompt. Peak RSS is the larger of dumpsys meminfo TOTAL RSS and VmHWM.

Use the CPU only, with the forked engine. It is the only configuration that fits a 3 GB budget. The engine drops the clean embedding-table pages every 8 tokens, which cut the peak from 2.97 to 2.65 GB with identical output. The GPU column is for reference: on this phone the GPU is slower than the CPU and needs more memory.

Use

The model expects the agent protocol, not free-form prompts. It reads with a 4k context and opens one fresh session per window of about 1,500 transcript tokens, with the compacted journal at its head. The model answers with at most 5 lines NOTE [time] (TYPE) text, then NEXT. See §4 and §12 of the archived mobile-v1 integration note.

With the forked engine. Use the fork vieenrose/LiteRT-LM, branch mobile-fused-attention, contrib/mobile_fused_attention/. Build it, then run:

mfa_engine --serve --dir mfa --main mfa/prefill_decode_fused.tflite --fused \
  --ctx 4096 --threads 8 --weight-cache model.wcache

The engine builds the XNNPACK weight cache on its first run, about 2.2 GB on disk.

With stock LiteRT-LM (reference only). Pass the .litertlm file with Backend.CPU(threadCount = 8) and maxNumTokens = 4096. This needs 4.6 GB.

About 1 statement in 9 is contradicted by the transcript, mostly the right figure attached to the wrong object. Check decisions and figures before relying on them.

No training data is included. The source recordings are subject to the Legislative Yuan IVOD terms of use. The weights are derived from Gemma 4 (Apache-2.0).

LoRA (qat_lora, safetensors)

lora/qat_lora.safetensors is the LoRA delta behind gemma-4-E4B-meeting-agent-zh-mobile-v1.litertlm (sha256 d4b626697750a1d90ad1602a49b89e34a9ad851e0dff500f18b25175a87ee5c1). It is applied to Google's Gemma-4-E4B mobile graph, not to the Hugging Face checkpoint, and it is in the training format (QAT), so it is not a standard PEFT adapter. Merge it into the mobile base to reproduce the LiteRT file; the file itself is the one to deploy.