Loading…
Loading…
distilbert/distilbert-base-uncased fine-tuned once on 5,000 queries of the BANKING77 train set. It is the W3 light path of FANN stage A, the cheaper model that the gate may route to. The model was trained on a separate split so that the calibration data and the test stream stay unseen.
On the 2,503-query risk-control split (a held-out part of the official train set, never used for training):
| Item | Value |
|---|---|
| Accuracy against the gold label | 0.9073 |
| Disagreement with the reference model (BERT-base) | 145 of 2,503 (0.0579) |
| One-sided upper bound on disagreement, significance 0.05 | 0.0662 |
The protocol calls a light path too strong if that bound is at most 1%. This one is above it. These are not accuracies on the official BANKING77 test set. The test set was not read.
| Item | Value |
|---|---|
| Base model | distilbert/distilbert-base-uncased, revision 12040accade4e8a0f71eabdb258fecc2e7e948be |
| Data | The first 5,000 queries of the official train set, shuffled with numpy.random.default_rng(20261030).permutation(10003) |
| Optimizer | AdamW, learning rate 5e-5, weight decay 0.01 |
| Schedule | Linear decay to 0, no warm-up |
| Batch size | 32 |
| Epochs | 10, fixed. No early stopping and no checkpoint selection. |
| Maximum length | 64 tokens |
| Seed | 20261030 |
| Hardware | One NVIDIA A10 (Modal), torch 2.6.0, transformers 4.48.3 |
The first run of this model with learning rate 2e-5 and 4 epochs reached 0.6904 accuracy. The recipe above replaced it, together with the BERT-base recipe, by a dated protocol amendment.
model.safetensors has sha256 8d3966629d225d0564853ff45d21c5d7596fe9570989a3a7053250e8bd856e67. manifest.json holds the recipe, the split hash, the epoch losses, and the package versions.
The rules are in the frozen FANN stage A protocol, v0.4.3: ic-research/fann-stage-a. The training code is in the quotient repository, recipes/finetune/banking77.
The 77 labels are the BANKING77 intent names, sorted alphabetically. Label names are in config.json.
Apache-2.0, as the base model. The BANKING77 data is CC BY 4.0 (PolyAI).
distilbert/distilbert-base-uncased