Stem GTE Memory

A custom GTE reranker fine-tuned for Stem, a personal assistant. It scores how relevant a stored memory fact, or a saved skill (a step-by-step procedure), is to a user's message. Training requests are in Czech, Slovak, German, and English; stored facts and skill descriptions are English.

This is the October 2026 q8 ONNX release, model ID gte-memory-20261021. It continues the September release (gte-memory-20260905-epoch2, Hub revision dae4f76d6974e70cdc007d3ca66d82226791822f, still downloadable by that revision) with training for skill selection, while fact scores were held to the September model. The five inference files total 358,119,414 bytes (341.53 MiB); the quantized graph alone is 341,032,790 bytes.

On 815 real assistant turns held out from training, scored in the Linux server image at Stem's thresholds, this release loaded 44 of 81 needed skills with a wrong skill loaded on 17 turns, where the September model with the same example requests loaded 42 with 35 wrong-load turns. Fact selection at the unchanged fact threshold stayed within a few percent of the September model. These are scoped results from one user's assistant, not universal model rankings.

Intended use

Rerank a shortlist of memory facts, or of skill descriptions and per-skill example requests, before an assistant responds. Fact and skill selection use separately calibrated thresholds.

Files and integrity

• onnx/model_quantized.onnx: selected q8 inference graph, ONNX opset 17.
• config.json, tokenizer.json, tokenizer_config.json, special_tokens_map.json: the exact measured inference configuration and tokenizer. The tokenizer files are byte-identical to the September release.
• stem-model.json: model identity, upstream and previous revisions, per-file SHA-256 hashes and sizes, and Stem thresholds.
• example.mjs: local Node.js inference example using Transformers.js.
• LICENSE, NOTICE and TRAINING-SOURCES.md: licensing, modification notices and the public training sources.

Pin a Hub commit revision when downloading, then verify all five inference files against stem-model.json. Hashes are also pinned in Stem's runtime. No training data, chat transcripts, benchmark examples, FP32 weights, or training checkpoints are included. config.json keeps its upstream auto_map metadata; the supported ONNX path below does not execute remote Python code.

Inference contract

Use Transformers.js PreTrainedModel, not its auto sequence-classification loader: this export retains the upstream custom model_type: new. This repository contains ONNX weights, not a PyTorch checkpoint.

1. Tokenize a (query, document) text pair with the supplied tokenizer.
2. Score one pair per forward pass, sequentially. Do not silently truncate. Stem rejects pairs longer than 8,192 tokens, including special tokens; long-context quality has not been validated here.
3. Read the single finite raw logit. Higher means more relevant. Do not apply sigmoid or reinterpret it as a probability.

node example.mjs /absolute/path/to/downloaded/model

For Stem's measured configuration:

These thresholds belong to this exact graph, tokenizer and pipeline. They are not interchangeable with the September release's skill thresholds, nor universal cutoffs.

Facts. Stem takes at most two preceding eligible user messages in chronological order, clipping each to 400 Unicode code points, and formats the query as:

Earlier messages from the user in this conversation:
- <earlier user message>
- <later user message>

Current message: <current user message>

With no eligible history, use the current message alone. Candidates come from an embedding shortlist of up to 24 facts.

Skills. The query is the current message; when it has fewer than 8 words, the previous user message (clipped to 400 code points) is prepended with a newline. Each shortlisted skill is scored as the maximum over separate pairs: name\ndescription and name\n<example request> for each of up to four example requests that Stem writes per skill. A skill without example requests uses its description pair and the description-only threshold. Stem shortlists four skills by embedding cosine (EmbeddingGemma 2) and loads at most two.

Training

Base: Alibaba-NLP/gte-multilingual-reranker-base, revision 8215cf04918ba6f7b6a62bb44238ce2953d8831c. Upstream implementation: Alibaba-NLP/new-impl, revision 40ced75c3017eb27626c9d4ea981bde21a2662f4.

September stage (unchanged, see revision dae4f76): 23,684 binary fact pairs from 88 fictional training profiles, query languages balanced across Czech, Slovak, German and English.

October stage, starting from the September epoch2 weights. Skill items were (message, skill) pairs, each holding every document the skill is scored on (description plus example requests), labeled 1 if the message needs that skill. Candidates were each message's top 8 skills by EmbeddingGemma 2 cosine plus any needed skill. Sources:

Facts were held to the September model by distillation: each step added 16 fact pairs from the September training set with the September model's own logits as targets (mean squared error, weight 20). The loss on skill items was binary cross-entropy on the maximum logit over the item's documents, matching how Stem scores skills, with positive weight 3. The encoder and classification head were trained and token embeddings frozen. AdamW, learning rate 1e-5, weight decay 0.01, batch size 16 skill items, 10% linear warmup, gradient clipping 1.0, one epoch (3,793 steps). Training queries longer than 320 tokens were truncated; inference does not truncate.

The checkpoint was chosen on development data only: 20% of the real conversations (held out by conversation) plus three whole held-out synthetic libraries (Czech/English, English, German). The 815-turn evaluation set below was never used for training or checkpoint selection; every conversation in it is later than, and disjoint from, the real training messages.

Export used ONNX opset 17 and the same dynamic int8 quantization settings as the September release (QInt8, per-channel, reduced range, symmetric weights, constant-matrix-only MatMul).

Evaluation and limitations

Skills, real turns. 815 turns from one user's assistant (27 Sep–20 Oct 2026; mostly Slovak and English, some Czech), each labeled by two blind LLM labelers with disagreements adjudicated by hand; 81 turns needed a skill that was in the library at the time. Both models used the same shortlist and the same example requests.

A paired bootstrap over conversations put the gain at about 5 more needed skills loaded at matched wrong-load budgets (probability of no gain about 0.02). Only 60 of the 108 needed skills reached the four-skill embedding shortlist; widening it to eight did not help with either model.

Skills, held-out libraries. Average precision over shortlisted (message, skill) items: Czech/English 0.873 → 0.952, English 0.839 → 0.921, German 0.856 → 0.928; held-out real conversations 0.263 → 0.401.

Facts. At the unchanged fact threshold, cost 3 × false negatives + false positives against the September model: development 171 vs 182, fresh synthetic 221 vs 224, real regression set 1: 162 vs 155, real regression set 2: 377 vs 375. Fact selection is not improved by this release, and set 1 is slightly worse.

Limits: the real evaluation comes from one user with one skill library, and labels are LLM judgments (human-adjudicated only where the two labelers disagreed). Rows without a platform were scored on macOS (Apple silicon, onnxruntime via Transformers.js 4.3.1). The same 12,000 pairs scored in the Linux x86-64 server image differed by a median of 0.07 logits but up to 8.4 on a few pairs (99th percentile 1.8); the September model differs as much (99th percentile 3.5). The skill thresholds were chosen where both platforms agree. Validate your target runtime. Long-input accuracy, final-answer quality and languages outside the four training languages were not evaluated.

License and attribution

Released under Apache License 2.0, following the upstream model's declared license. See LICENSE, NOTICE and TRAINING-SOURCES.md. Stem modifications comprise memory- and skill-relevance fine-tuning, ONNX export, quantization and task-specific calibration. This release is a Stem adaptation, not an official Alibaba release.