gliner-probe-smoke

Fine-tune of urchade/gliner_large-v2.1 for data-use mention extraction (dataset / survey / census / registry mentions in economics research papers).

Labels

NAMED_DATA — a proper name, title, or acronym of a specific data source
DESCRIPTIVE_DATA — a source described in words but not named
VAGUE_DATA — generic data wording with no identifiable source

Training

base model: urchade/gliner_large-v2.1
dataset: rafmacalaba/usage-sensitivity-probe (gliner config)
corpus: all
epochs: 1
learning rate: 5e-06
batch size: 16
precision: bf16

Evaluation (holdout)

thr	tp	fp	fn	precision	recall	f0.5	f1
0.10	36	865	4	0.0400	0.9000	0.0494	0.0765
0.20	33	445	7	0.0690	0.8250	0.0845	0.1274
0.30	29	243	11	0.1066	0.7250	0.1285	0.1859
0.40	22	142	18	0.1341	0.5500	0.1580	0.2157
0.50	16	75	24	0.1758	0.4000	0.1980	0.2443
0.60	6	35	34	0.1463	0.1500	0.1471	0.1481
0.70	3	18	37	0.1429	0.0750	0.1210	0.0984

Best F0.5: 0.1980 (thr=0.5) Best F1: 0.2443 (thr=0.5)

Evaluation breakdown (holdout)

group	examples	spans	thr	precision	recall	f0.5	f1
overall	64	40	0.50	0.1758	0.4000	0.1980	0.2443
prwp	14	12	0.50	0.1212	0.3333	0.1389	0.1778
fcv	50	28	0.50	0.2069	0.4286	0.2308	0.2791
general_prwp	14	12	0.50	0.1212	0.3333	0.1389	0.1778
fcv_pads_east_africa	27	11	0.50	0.1481	0.3636	0.1681	0.2105
jdc_operational	6	5	0.70	1.0000	0.2000	0.5556	0.3333
refugee_pads	5	3	0.60	1.0000	0.3333	0.7143	0.5000
reliefweb	12	9	0.50	0.3077	0.4444	0.3279	0.3636

Per-label (overall)

label	examples	spans	thr	precision	recall	f0.5	f1
NAMED_DATA	64	9	0.60	0.1200	0.3333	0.1376	0.1765
DESCRIPTIVE_DATA	64	10	0.30	0.0149	0.1000	0.0180	0.0260
VAGUE_DATA	64	21	0.50	0.1154	0.1429	0.1200	0.1277

Origin breakdown (per-origin metrics)

origin	examples	spans	thr	precision	recall	f0.5	f1
fcv_pads_east_africa	27	11	0.50	0.1481	0.3636	0.1681	0.2105
general_prwp	14	12	0.50	0.1212	0.3333	0.1389	0.1778
jdc_operational	6	5	0.70	1.0000	0.2000	0.5556	0.3333
refugee_pads	5	3	0.60	1.0000	0.3333	0.7143	0.5000
reliefweb	12	9	0.50	0.3077	0.4444	0.3279	0.3636

Per-label details

label	examples	spans	thr	precision	recall	f0.5	f1
NAMED_DATA	64	9	0.60	0.1200	0.3333	0.1376	0.1765
DESCRIPTIVE_DATA	64	10	0.30	0.0149	0.1000	0.0180	0.0260
VAGUE_DATA	64	21	0.50	0.1154	0.1429	0.1200	0.1277