Job Posting Skill Extractor (DeBERTa-v3 + CRF)

Extracts skills (e.g. "mentor junior engineers") and knowledge (e.g. "Python", "AWS") spans from English job postings. Skill and knowledge spans can overlap.

• Demo: https://huggingface.co/spaces/srish327/skill-extractor (runs in your browser, using onnx/model_quantized.onnx: 271 MB, test F1 0.6309 vs 0.6318 for fp32)
• Code, training, experiments & CI: https://github.com/MSR327/skillspan-ner

Usage

The model has two BIO tagging heads (6 outputs: SKILL O/B/I + KNOWLEDGE O/B/I) and learned CRF transitions stored in skillex_crf.json, so use the skillex decoder rather than the stock pipeline:

# pip install git+https://github.com/MSR327/skillspan-ner.git
from skillex.predict import SkillExtractor
ex = SkillExtractor("srish327/deberta-v3-skillspan-crf")   # picks up the CRF automatically
ex.extract("We need a data engineer with strong SQL and stakeholder management skills.")

Results (SkillSpan test set, exact span-match F1)

This checkpoint (seed 43, chosen as the best seed on validation; test evaluated once): F1 0.6316 (skill 0.5371, knowledge 0.7178).

Same setup across 3 seeds, compared with alternatives (mean ± std):

The paper's combined score follows Benikova et al. (2014), so the comparison is close but not identical. Our JobBERT run reproduces the paper's number (0.598 vs 0.597).

Training

• DeBERTa-v3-base encoder with two linear BIO heads, one for SKILL and one for KNOWLEDGE. Two heads are needed because 4.4% of knowledge spans sit inside a skill span.
• Each head has a linear-chain CRF over first-subword positions. O → I is hard-forbidden. Loss is the sum of the two heads' CRF negative log-likelihoods.
• 5 epochs, lr 3e-5, batch 32, warmup 10%, weight decay 0.01, max length 256, fp32 weights with fp16 autocast.
• Best epoch chosen by validation CRF F1. One Kaggle T4, about 8 minutes.

Limitations

• Trained on English tech and house-job postings only (SkillSpan). Expect lower recall in other domains.
• Skill spans are long and fuzzy: boundaries for skills (F1 ~0.56) are much less reliable than for knowledge (~0.71). Human annotator agreement on this data is κ ≈ 0.70 to 0.75.
• Not intended for automated hiring decisions.

Citation

Zhang, Jensen, Sonniks, Plank. SkillSpan: Hard and Soft Skill Extraction from English Job Postings. NAACL 2022.