Gemma-3-270M meeting agent (zh-TW): LiteRT, smoke-test model
Gemma-3-270M-it fine-tuned on the same reading-agent task and protocol as Luigi/gemma-4-E4B-meeting-agent-zh-LiteRT: it reads a live zh-TW meeting transcript window by window and writes short cited notes.
This model is not accurate enough for real use. More than half of its notes are contradicted by the transcript. It exists to test an integration quickly: it speaks the same protocol and note format as the E2B/E4B models, loads in a fraction of the memory and runs much faster, so an app pipeline (model loading, windowing, journal compaction, note parsing) can be exercised end to end in seconds.
Code, agent protocol and evaluation: https://github.com/vieenrose/meeting-summarizer. Integration: integration note §12–§13.
| file | |
|---|---|
gemma-3-270m-meeting-agent-zh-q8.litertlm |
int8 weights (channelwise, embedding included), 4k KV cache (289,698,480 bytes, sha256 dae7b572c67ef38465d52716d80f4830c9807607b65f75d20f35e05b14620b63) |
system_prompt.txt |
the system prompt the model was trained with; use it verbatim |
How it was built
- Fine-tuning. Full fine-tune of
google/gemma-3-270m-iton the reading-agent turns used for the E4B model, with turns holding a contradicted or unsupported note masked out. - Quantization-aware training. The weights were then trained with LiteRT's int8 grid in the loop (per-channel weights, per-token dynamic activations), distilling from the float copy of the same model.
- Export.
litert_torchwithdynamic_int8, KV cache 4096, prefill signatures 128 and 1024, packed with the Gemma 3 chat template and SentencePiece tokenizer.
An int4 export (154 MB) was also tried. Even with quantization-aware training it lost a quarter of the coverage, so only int8 is published.
Results
Measured on 13 held-out sessions of Legislative Yuan committee meetings transcribed by on-device ASR, with a 4k context; the judge is Gemma-4-31B, which checks each cited statement against the transcript.
| this model (LiteRT int8, QAT) | same model, LiteRT float32 | same model, int8 without QAT | E4B (reference) | |
|---|---|---|---|---|
| notes contradicted by the transcript | 58 % | 57 % | 61 % | 11 % |
| coverage of the reference key points | 0.64 | 0.65 | 0.58 | 0.93 |
| decision recall | 51 % | 49 % | 44 % | 73 % |
Quantization-aware training brings the int8 model back to the level of the float model. With 13 sessions, differences of a few points are within noise. The GGUF Q8_0 of the same weights run with llama.cpp scores about the same (56 % contradicted, coverage 0.72).
Use
Same as the E4B model: 4k context, one fresh session per window of about 1,500 transcript tokens with
the compacted journal at its head; the model answers with at most 5 lines NOTE [time] (TYPE) text,
then NEXT. With stock LiteRT-LM: Backend.CPU, maxNumTokens = 4096.
Stop generation at NEXT yourself and cap replies at about 400 tokens: this small model often keeps
writing after NEXT, and the litert-lm serve OpenAI endpoint ignores stop and max_tokens
(it only reads max_completion_tokens).
No training data is included. The source recordings are subject to the Legislative Yuan IVOD terms of use. The weights are derived from Gemma 3 and are distributed under the Gemma Terms of Use.