mdg-nlp/gpt-2-timex-sentence-recognition

Source: 🤗At risk🏴‍☠️ Rescued by: isilicontoken-classificationapache-2.0124M params503 MBsafetensors✓ 5 checksumsupdated 46d ago

Magnet 0

GPT-2 Fine-Tuned for Time Expression Extraction (TIMEX)

Overview

This model is a fully fine-tuned version of openai-community/gpt2 for Time Expression Extraction (TIMEX).

The model identifies explicit temporal expressions in natural language text, including dates, times, durations, frequencies, and other temporal references.

The task is formulated as token-level sequence labeling using BIO tags.

Label Map

ID	Label	Description
0	O	Token is outside a temporal expression
1	B-T	Beginning of a temporal expression
2	I-T	Inside a temporal expression

Task

Time Expression Extraction (TIMEX)

Given an input sentence, the model identifies spans corresponding to temporal expressions.

For example:

Input:
The conference started on April 9, 1997 and lasted for three days.

Temporal expressions:
April 9, 1997
three days

Time expression extraction provides structured information about references to time in text and can serve as a component in broader temporal understanding and reasoning systems.

Model

GPT-2 is a decoder-only transformer architecture. This model uses GPT-2 with a token-classification head and is fully fine-tuned for sentence-level temporal expression extraction.

The model is one of the fine-tuned pretrained language model baselines used to study multi-dimensional generalization in temporal extraction tasks.

Dataset

Dataset: mdg-nlp/timex-recognition-sentence-original

Labels

0 -> O
1 -> B-T
2 -> I-T

Training Details

Configuration	Value
Base Model	`openai-community/gpt2`
Task	Time Expression Extraction
Formulation	Token Classification
Label Scheme	BIO
Fine-tuning	Full fine-tuning
Epochs	3
Learning Rate	2e-5
Optimizer	AdamW
Weight Decay	0.01
Warmup Steps	500
Train Batch Size	16
Evaluation Batch Size	32
Max Sequence Length	128

Evaluation

All reported scores use micro-F1.

Evaluation Setting	Micro-F1
Base	0.88
T2C	0.48
Adversarial (Adv)	0.86
Length	0.83
Vocabulary (Voc)	0.88
Adversarial Perturbation (Adv_P)	0.74

The evaluation settings measure generalization across the original task, cross-domain data, adversarial examples, structural changes, vocabulary variation, and adversarial perturbations.

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification

model_id = "mdg-nlp/gpt-2-timex-sentence-recognition"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

text = "The conference started on April 9, 1997."

inputs = tokenizer(text, return_tensors="pt", truncation=True)
outputs = model(**inputs)

predictions = outputs.logits.argmax(dim=-1)[0].tolist()

id2label = {
    0: "O",
    1: "B-T",
    2: "I-T"
}

tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])

for token, prediction in zip(tokens, predictions):
    print(token, id2label[prediction])

Limitations

Performance may decrease under distribution shifts.
Cross-domain temporal expressions may be more difficult to identify.
Adversarial perturbations can affect extraction performance.
Tokenization can affect predicted expression boundaries.
Longer or structurally different inputs may produce different behavior.
The model extracts temporal expressions and does not directly perform temporal relation classification.

Research Context

This model was developed as part of research on multi-dimensional generalization in temporal extraction tasks.

The broader work studies temporal and event expression extraction under different forms of distribution shift, including domain shift, adversarial perturbations, vocabulary variation, compositional changes, and structural changes.

Contact

Author: Arnav Verma
Affiliation: New York University (2026–Present), University of North Texas (2023–2026)

Citation

If you use this model in your research, please cite the associated work when available.