OTel-LLM-12B-IT

OTel-LLM-12B-IT is a context-grounded telecom language model full-parameter fine-tuned on OTel telecommunications data. It is part of the OTel Family of Models, an open-source initiative to build reference AI resources for the global telecommunications sector.

Across the 17 core OTel LLM baselines, OTel fine-tuning improves context-grounded correctness over the base checkpoints by +3.3 to +9.4 percentage points.

Community Use

As of May 3, 2026, the released OTel models had been downloaded over 16 million times, and the Open Telco AI project had received 157+ pieces of media coverage worldwide.

Model Details

Model Lineage

google/gemma-3-12b-it -> OTel-LLM full-parameter post-training -> farbodtavakkoli/OTel-LLM-12B-IT

OTel vs. Base Model

Standard errors are computed with bootstrap resampling (n=1000) over the held-out OTel evaluation partition. LLM correctness is the average score from two judge models, GPT-4o mini and Claude Sonnet 3.5, each judging the answer against the retrieved context and reference answer.

Evaluation Caveats

• LLM results measure context-grounded answer generation from retrieved context, not unrestricted context-free telecom QA.
• Reported standard errors come from bootstrap resampling over the held-out evaluation partitions.
• Answer quality depends on the retriever, reranker, context window, and prompt policy around the model.
• External benchmark transfer, multilingual performance, and per-subdomain performance should be evaluated separately for production settings.

Training Data

The model was trained on telecom-focused data curated by 100+ domain experts. The raw corpus contained roughly 1.1M training points and was filtered to 326,767 higher-confidence examples.

Released datasets: OTel-LLM, OTel-Embedding, OTel-Reranker, and OTel-Safety.

The OTel datasets release derived QA/retrieval/reranking examples rather than the raw source documents.

Each released dataset includes a dataset card and Croissant metadata with Responsible AI fields for data limitations, biases, sensitive-information considerations, use cases, social impact, synthetic-data status, and provenance.

Representative Training Row

OTel-LLM rows pair a context-grounded telecom RAG prompt with a reference completion.

{
  "anchor": "How can a cell be considered to be operating in MBSFN mode for 3.84/7.68 Mcps TDD?",
  "completion": "A cell shall be considered to be operating in MBSFN mode when individual scrambling codes are assigned to all timeslots via the IE \"TDD MBSFN Information\".",
  "abstention": false,
  "n_positive_chunks": 1,
  "n_negative_chunks": 4
}

Intended Use

This model is intended for context-grounded telecom answer generation in Retrieval-Augmented Generation (RAG) pipelines. It should receive retrieved telecom context and generate an answer grounded in that context.

The model is not optimized for unrestricted context-free question answering. For questions where the retrieved context is missing or insufficient, use an abstention-aware prompt or one of the dedicated -Safety variants.

Training Recipe

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "farbodtavakkoli/OTel-LLM-12B-IT"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

prompt = """You are a precise telecom assistant in a RAG pipeline.
Use only the retrieved context to answer.

User Question
What is the purpose of the F1 interface in O-RAN?

Retrieved Contexts
CONTEXT 1
The F1 interface connects the O-RAN Distributed Unit (O-DU) to the O-RAN Central Unit (O-CU).

Answer:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations and Responsible Use

• OTel models are domain-specific to telecommunications and should not be treated as general-purpose models.
• The current release is English-only and primarily text-centric.
• The reported OTel performance results use held-out OTel evaluation partitions and should not be interpreted as results from a fully independent external benchmark suite.
• Aggregate scores can hide subdomain variation; collaborator stress tests suggest O-RAN retrieval is comparatively strong, while academic-paper and GSMA PRD examples need further curation.
• Generated telecom content should be verified before operational, customer-facing, regulatory, safety, or network-configuration use.
• Users must comply with both the OTel release license and the upstream base-model license or terms.
• For unrestricted telecom QA without retrieved context, use a separately evaluated context-free QnA model rather than assuming this RAG-oriented checkpoint will behave optimally.

Related Models

Language Models

• OTel LLM 2.0 Collection
• OTel LLM 1.0 Collection

Embedding Models

• OTel Embedding Collection

Reranker Models

• OTel Reranker Collection

Resources

• Project Page
• OTel code
• OTel Selected Coverage
• OTel Paper
• OTel 1.0 Coverage
• OTel 2.0 Coverage
• OTel Docker

Citation

@inproceedings{tavakkoli2026otel,
  title     = {OTel: Open Telco AI Datasets, Benchmarks, and Models},
  author    = {Tavakkoli, Farbod and others},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Evaluations and Datasets Track},
  year      = {2026},
  url       = {https://github.com/farbodtavakkoli/OTel}
}

Contact

If you have any technical questions, please feel free to reach out to farbod.tavakkoli@att.com or farbodtavakoli@gmail.com