GLiNER2.5 base (gliner2.5-base-v1) — LiteRT (Android)

A LiteRT conversion of fastino/gliner2.5-base-v1 (revision 1a8bc24e00dc7300b9017c81d63e3dcdabb26596, Apache-2.0, by Fastino), converted by spybyscript. It carries every trained head: entities, classification, structures, relations and records. gliner2's own code decodes the outputs. For the model itself (training, labels, benchmarks, intended use), see the original model card, copied here as model-card.md. The split into graphs follows FluidInference's Core ML port (gliner2-5-base-coreml, Apache-2.0).

Files

config.json lists every graph's signatures and shapes.

Use

# pip install gliner2==2.0.0 torch numpy ai-edge-litert
import sys; sys.path.insert(0, "runtime")
from litert_boundary_runtime import LiteRTBoundaryExtractor
from gliner2 import Schema

model = LiteRTBoundaryExtractor(".")          # this repository's folder
model.extract_entities("Alice founded Acme in Toronto in 2020.", ["person", "organization", "city"])
model.extract_relations("Alice founded Acme in Toronto.", ["founded", "located_in"])
schema = Schema().entities(["person", "organization"]).relations(["works_for"])
model.extract("Ravi Shah is an engineer at Tidewater Systems.", schema, include_confidence=True)

Every public gliner2 call works: extract, extract_entities, classify_text, extract_json and extract_relations, plus their batch_* and *_long forms. The runtime runs gliner2's BoundaryExtractor with every trained module on LiteRT. Schema handling, the candidate pool and the decoders are the library's own code.

On a phone, run the same graphs with LiteRT's CompiledModel; the runtime shows the host steps.

• The encoder is the only graph worth an accelerator. Use the NPU with your chip's precompiled file from npu/ (or fp16hw, which the phone compiles in about 6 s), the GPU with wfp16 at FP32 (or fp16hw at FP16 where it stays finite), or the CPU.
• The six head graphs run on the CPU in 1–16 ms each.

Fixed sizes. The original runs with dynamic shapes; these are FluidInference's bucket sizes. One request holds at most:

• 128 subword tokens (schema and text) and 64 text words;
• 8 queries (entity labels, structure fields, relation roles) and 8 classification choices;
• a candidate pool of 192 (the checkpoint's own);
• 4 relation types with 256 head/tail pairs;
• 64 explicit spans per query;
• records of up to 8 fields with 192 candidates each.

A larger request is refused with a ValueError, never truncated. Run long text through the *_long calls, for example extract_entities_long(text, labels, chunk_size=40, chunk_overlap=8). The weights change only in storage and compute precision.

Validation

runtime/verify_parity.py runs each request through gliner2 with the original weights and through the reference runtime (LiteRT CPU), then compares the structured results:

• The corpus (runtime/parity_corpus.json): 296 requests over 16 call types (entities, relations, structures, the three record modes, attributes, single- and multi-label classification, mixed schemas). All 296 are identical, with span F1 1.0 and confidences within 0.0014.
• FluidInference's eleven schema-path fixtures: 11 of 11.

validation-full.json and validation.json hold the numbers.

Phones

LiteRT 2.3.2 with qnn-runtime 2.50.0. Each graph ran on a request recorded from the corpus: 10 calls after a rest, at thermal status 0. The CPU ran 4 threads; the GPU ran at NORMAL priority, timed as whole calls. The NPU compiled the graphs on the phone; for the encoder that takes about 6 s, once.

— means LiteRT's GPU does not support the graph; run it on the CPU.

• The precompiled encoder (npu/qualcomm/gliner2_base_encoder_fp16hw_SM8850.tflite) on the Fold: 8.1 ms a call, loaded in 0.2 s, with 31 of 32 recorded requests unchanged (below), as with the encoder the phone compiled itself.
• A request with the encoder on the NPU and the heads on the CPU takes about 15 ms on the Fold and 20 ms on the S23. On the CPU alone it takes about 45 ms on the Fold and 150 ms on the S23.
• Answers with the phones' encoder outputs. 32 recorded requests, two per call type, ran through the rest of the pipeline:
  • GPU FP32: unchanged on both phones;
  • NPU: 31 of 32 on the Fold and 30 of 32 on the S23 (only latent-record requests changed);
  • fp16hw at GPU FP16: 31 of 32 on the Fold. On the S23's Adreno 740 it gives non-finite outputs, with or without FP32 accumulation, so use FP32 there.
• Dirty memory, encoder: about 0.45 GB on the CPU, 0.6 GB on the GPU at FP16 and 0.85 GB at FP32. The NPU holds about 0.5 GB after compiling on the phone.
• GPU priority depends on the GPU. LOW priority is meant to let other GPU work go first. On the Fold's Adreno 840 it keeps an app's rendering smooth; on the S23's Adreno 740 it does not, and only slows each call. The measurements are with GLiNER2.5-Decide's encoder: see spybyscript/gliner25-decide-litert.

Qualcomm NPU (npu/)

Runtime:

• LiteRT 2.3.2 with litert-npu-runtime-qualcomm 2.3.2 and Qualcomm's qnn-runtime 2.50.0 (Qualcomm's AI Stack License);
• files precompiled on x86-64 Linux with QAIRT 2.50.0, each in one NPU partition (930 of 930 operators);
• a file runs only on its chip (Build.SOC_MODEL) and with these runtime versions.

Each file is gliner2_base_encoder_fp16hw.tflite (signature t128) compiled ahead of time, so the phone loads it without compiling. npu/manifest.json lists every file's chip, size and SHA-256. Keep the head graphs on the CPU.

Files

• LICENSE (Apache-2.0) and NOTICE.txt
• model-card.md, the original card
• config.json, the contract
• validation.json and validation-full.json
• SHA256SUMS
• tokenizer/
• source/ (the checkpoint's configs)
• runtime/
• npu/ (the precompiled encoder per chip and npu/manifest.json)