HiGliZNet

Hierarchical zero-shot classification over the UNSPSC product taxonomy, extending GliZNet with a beam-search training strategy that mirrors inference exactly.

Quick Start

from higliznet.predictor import HierarchicalClassificationPipeline
from higliznet import HiGliZNetForSequenceClassification
from higliznet.tokenizer import HiGliZNETTokenizer
from higliznet.data_utils import TaxonomyDB

model     = HiGliZNetForSequenceClassification.from_pretrained("alexneakameni/higliZnet-deberta-v3-base")
tokenizer = HiGliZNETTokenizer.from_pretrained("alexneakameni/higliZnet-deberta-v3-base")
taxonomy  = TaxonomyDB.load("unspsc_taxonomy.json")  # bundled in the repo

pipeline = HierarchicalClassificationPipeline(
    model=model, tokenizer=tokenizer, taxonomy=taxonomy,
    beam_width=5, max_candidates=10,
)

results = pipeline("Hydraulic excavator bucket teeth", top_k=3)
for r in results:
    print(r)
# PathPrediction('Mining and drilling > ... > Excavating machinery > ...', score=0.87)

---

Overview

UNSPSC is a 5-level taxonomy with 13 293 nodes (UNGM May 2026 release):

Training signal comes from UNSPSC commodity titles (L4 descriptions projected up the ancestor chain).
Zero-shot generalisation is measured on GSIN procurement language (real purchase descriptions never seen during training).

---

Architecture

HiGliZNetForSequenceClassification wraps microsoft/deberta-v3-base (~189 M params).

Sequence format

Each candidate is shown with its direct children as in-context examples ([CON] token):

[CLS] query_text [SEP]
  candidate_1 [CON] child_1 [CON] child_2 [LAB]
  candidate_2 [CON] child_1 [LAB]
  ...

Identical format at training and inference — no train/inference mismatch.

Training strategy (HierarchicalBeamDataset)

Each training record {"text": str, "positive_codes": [L0, L1, …, L4]} generates one sequence per level in the positive path. These N sequences are batched into [B, N, max_seq] tensors and processed in a single backbone call via .view(B×N, max_seq).

Loss

GliZNet's standard combined loss (BCE + SupCon + Label Repulsion) applied independently at each level. No path-consistency term — the hierarchy is implicit in the beam-search training strategy.

Inference (HierarchicalClassificationPipeline)

Top-down beam search through the taxonomy, one forward pass per level:

L0: score 10 segment groups   → keep all, prune after L1 expansion
L1: score children of top-B   → keep top-B globally
L2: score children of top-B   → keep top-B globally
L3: score children of top-B   → keep top-B globally
L4: score children of top-B   → keep top-B globally

Every output path is a valid root-to-leaf walk — structurally inconsistent predictions are impossible.

---

Data

---

Train from Scratch

uv sync --all-groups --all-extras
bash scripts/train_higliZnet.sh

Logs → nohup.out. Checkpoints → artefacts/higliZnet_<TIMESTAMP>/.

---

Key Hyperparameters

---

Evaluation

Zero-shot beam-search evaluation on GSIN procurement descriptions (L3 Class target):

---

Project Structure

train_higliznet.py           # HuggingFace Trainer entry-point
scripts/train_higliZnet.sh   # Launch script
higliznet/
  model.py                   # HiGliZNetForSequenceClassification
  data.py                    # HierarchicalBeamDataset + hierarchical_beam_collate_fn
  data_utils.py              # TaxonomyDB, build_unspsc_taxonomy_db, build_gsin_records
  tokenizer.py               # HiGliZNETTokenizer ([LAB], [CON] special tokens)
  predictor.py               # HierarchicalClassificationPipeline (beam search)
  arguments.py               # HiModelArgs
  metrics.py                 # ancestor_precision, path_recall, hierarchical_f1
zero-shot-classification/    # Base GliZNet components
data/
  UNGM_UNSPSC_25-May-2026.xlsx
  gsin_to_unspsc_mapping.csv
artefacts/                   # Training outputs (checkpoints, W&B logs)
notebooks/
  higliZNet_v2_walkthrough.ipynb
  gsin_eval_walkthrough.ipynb