OTel-2.0-LLM-31B-IT-QLoRA

Versioning notice: This card describes the current OTel 2.0 QLoRA checkpoint. The model weights may be updated over time. For reproducible evaluation or production deployment, pin a specific model revision, checkpoint hash, or release tag rather than relying on the floating latest version.

OTel-2.0-LLM-31B-IT-QLoRA is a telecom-specialized instruction model post-trained from Gemma 4 31B-IT on approximately 440 billion telecom training tokens. It is the first release in the OTel 2.0 family and is designed to support telco-grade AI workflows across network operations, standards interpretation, product development, network configuration assistance, RAG, and telecom-specific question answering.

This checkpoint was produced with parameter-efficient fine-tuning: a LoRA (QLoRA) adapter trained on the 4-bit (bitsandbytes NF4) quantization of Gemma 4 31B-IT and released as a merged, standalone bf16 model. The NF4 base is dequantized to bf16 and the LoRA deltas folded in, yielding a full-precision HuggingFace checkpoint (Gemma4ForConditionalGeneration, quantization_config: null) with no inference-time dependency on bitsandbytes, PEFT, or Unsloth. See Adapter and Merge Details.

This is the QLoRA variant of the OTel 2.0 31B release. Unlike the full-parameter OTel-2.0-LLM-31B-IT, the Gemma 4 31B-IT base was frozen in 4-bit (bitsandbytes NF4) during training and only a bf16 LoRA adapter was trained on top. This makes training substantially faster and cheaper, but adapting a quantized base can yield lower quality than full-parameter bf16 fine-tuning. The released weights are full bf16 (the 4-bit base is dequantized and the adapter folded in) — the -QLoRA suffix refers to how the model was trained, not to the precision of the released checkpoint, which is not a 4-bit inference model.

OTel 2.0 extends the original OTel effort from a RAG-oriented telecom fine-tuning release into a larger domain-adapted training program. The model was trained from a much larger standards and telecom corpus, with new data preparation coverage for direct telecom QnA, abstention, RAG, base-model-style telecom data, and general-purpose instruction-following and tool-calling examples. The current training mixture does not include telecommunications-specific MCP, tool-calling, or instruction-following examples.

Release Status

This repository contains the OTel 2.0 QLoRA checkpoint. Its weights may be updated as model development continues. Use a pinned model revision, checkpoint hash, or release tag for reproducible evaluation and production deployment.

Model Details

Model Lineage

Gemma 4 31B-IT -> 4-bit NF4 quantization -> OTel 2.0 telecom data processing and QLoRA post-training -> merge (dequantize NF4→bf16 + fold LoRA) -> OTel-2.0-LLM-31B-IT-QLoRA

What Changed From OTel 1.0

Relative to OTel 1.0, OTel 2.0 increases raw source coverage by roughly 25x and training-token volume by roughly 440x. More importantly, the data mixture is broader: OTel 1.0 focused primarily on context-grounded RAG and abstention, while OTel 2.0 adds direct telecom knowledge QnA, general-purpose instruction-following and tool-use-oriented examples, and base-model-style telecom training data. Telecommunications-specific MCP, tool-calling, and instruction-following examples are not part of the current training mixture.

Training Data

The starting corpus contains approximately 15 billion raw tokens provided by GSMA through Open Telco AI. The corpus includes telecom standards and technical material from seven standards development and industry organizations:

Dense technical specifications from these sources were converted into material suitable for model training. The raw corpus was processed into over 1 trillion tokens using Red Hat's open-source Synthetic Data Generation Hub (SDG Hub) on Microsoft Azure Managed Compute, using approximately 530 GPUs, primarily AMD MI300X. From this processed pool, approximately 440 billion training tokens were generated for OTel 2.0 post-training. Model training ran on on-premises AMD MI355X GPUs with Dell Technologies infrastructure and servers.

Data Preparation Capabilities

OTel 2.0 includes a broader supervised and synthetic-data mixture than OTel 1.0:

Intended Use

OTel-2.0-LLM-31B-IT-QLoRA is intended for telecom-focused applications where domain knowledge, standards familiarity, and deployment control matter. Suitable use cases include:

• Retrieval-Augmented Generation over telecom standards, specifications, and internal technical documentation.
• Standards interpretation and summarization for 3GPP, ETSI, GSMA, CAMARA, ITU, O-RAN, and TM Forum materials.
• Product development, network configuration assistance, and engineering support tasks.
• Telecom-specific direct QnA where the model has been separately evaluated for the target benchmark or application.
• Agentic workflows where a larger system provides verified tools, retrieval, validation, and audit logging.

For high-impact operational use, the model should be deployed with retrieval, source attribution, validation checks, and human review appropriate to the system risk.

Adapter and Merge Details

This model was produced with parameter-efficient fine-tuning (QLoRA) and released as a merged bf16 checkpoint.

• Training: a LoRA adapter (r=256, alpha=512, dropout 0.0, all-linear targets) trained on the 4-bit NF4 base unsloth/gemma-4-31b-it-unsloth-bnb-4bit with Unsloth. During training the base is dequantized from NF4 in the forward pass, so the effective base is dequant(nf4(W)).
• Merge for release: to preserve accuracy, the adapter is folded into the dequantized 4-bit base (dequant(nf4(W)) + B·A), not the pristine bf16 base. The adapter learned to correct the quantized base, so merging onto the pristine bf16 base degrades quality; the correct procedure dequantizes each Linear4bit to bf16 first and then folds the deltas (note that calling merge_and_unload() directly on a 4-bit base re-quantizes the result).
• Result: a standard HF checkpoint (Gemma4ForConditionalGeneration, quantization_config: null, ~62.5 GB) loadable with AutoModelForCausalLM.from_pretrained — no bitsandbytes, PEFT, or Unsloth needed at inference.

Serving with vLLM

Requirements

• A recent vLLM with Gemma 4 support. Gemma 4 uses a heterogeneous per-layer attention layout (some layers have a larger head dimension); vLLM builds without the Gemma 4 loader fail at weight load with a shape mismatch such as Attempted to load weight (512) into parameter (256). If you hit this, upgrade to a vLLM release that lists Gemma 4 support. vLLM pins its own transformers/torch, so a matching stack is installed for you.
• 2 GPUs with tensor parallelism: the bf16 weights are ~62 GB and do not fit on a single 80 GB GPU alongside the KV cache. On larger-memory GPUs a single card may suffice.

Serve

# 2-GPU tensor parallelism; adjust --max-model-len / --gpu-memory-utilization to your hardware
CUDA_VISIBLE_DEVICES=0,1 vllm serve <model_dir_or_hf_repo> \
  --served-model-name otel-2.0-llm-31b-it-qlora \
  --host 0.0.0.0 --port 8000 \
  --tensor-parallel-size 2 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.90

Verify

curl http://localhost:8000/v1/models      # -> otel-2.0-llm-31b-it-qlora

Inference

Query with /v1/completions using a prompt built from the chat template (see Prompt Format). Use add_special_tokens: false (the chat template already prepends <bos>; a second one corrupts the prompt), and stop token ids [1, 106] (<eos> and end-of-turn). For deterministic output use greedy decoding (temperature: 0).

curl http://localhost:8000/v1/completions -H 'Content-Type: application/json' -d '{
  "model": "otel-2.0-llm-31b-it-qlora",
  "prompt": "<bos><|turn>user\nWhat model are you?<turn|>\n<|turn>model\n",
  "max_tokens": 256,
  "temperature": 0,
  "add_special_tokens": false,
  "stop_token_ids": [1, 106]
}'

Prompt Format

OTel 2.0 uses the Gemma 4 chat template. Build prompts with apply_chat_template(messages, add_generation_prompt=True); when sending a raw prompt to /v1/completions, trim it to end exactly at the <|turn>model\n marker so the model generates the reply directly.

No system prompt is required. The model works directly from the user turn, including knowing its own identity:

Input:
What model are you?

Output:
I'm OTel 2.0, a large language model trained by AT&T Chief Data Office.

You can pass your own system prompt as the first message when your application needs one; it is used as-is with no special handling.

Limitations and Responsible Use

Supported Language and Modality

OTel 2.0 is post-trained and supported for English-language text interactions.

The released checkpoint is architecturally multimodal: it is a Gemma4ForConditionalGeneration model and retains the full Gemma 4 vision encoder inherited from the base model, so it will structurally accept image inputs alongside text. However, only the text/language pathway was trained. OTel 2.0 post-training used a LoRA adapter applied exclusively to the text/language layers on a text-only telecom corpus; the vision encoder was frozen and carries the untrained, stock Gemma 4 weights with no OTel/telecom adaptation.

As a result:

• Text is the trained, intended, and evaluated modality.
• Image inputs are handled only by the inherited base-model vision path. This capability has not been trained, tuned, or evaluated for OTel 2.0, and no claim is made about its quality or safety for any use. Treat image handling as untested base-model behavior, not an OTel 2.0 feature.
• OTel 2.0 has not been established as a multilingual model, and non-text output modalities (audio, video) are not supported.

For production use, drive the model with English text; do not rely on image inputs without your own task-specific evaluation.

Tool Use and Instruction Following

The training data includes general-purpose tool-calling and instruction-following examples, but it does not currently include:

• Telecommunications-specific MCP examples.
• Telecommunications-specific tool-calling examples.
• Telecommunications-specific instruction-following examples.

General-purpose tool-use training should not be interpreted as readiness to operate telecommunications tools autonomously. This includes multi-step tasks where a scorer measures the resulting network state rather than the text of a response; general-purpose tool-use training does not establish competence at such closed-loop operational workflows (see Autonomous Operational Task Completion Not Benchmarked). Agentic deployment requires an external tool runtime, validated tool schemas, permission controls, audit logging, safeguards, and human review appropriate to the risk. The model alone does not guarantee correct or safe tool execution.

Telecommunications Data Not Included in Training

The current training mixture does not include the following classes of operational telecommunications data:

• Event data: Elasticsearch records covering user activity, anomalies, failures, IMS events, and RADIUS authentication.
• Network-performance data: KPIs, 5G performance metrics, and Passive Intermodulation (PIM) interference data.
• RF and spectral data: Field-test results, antenna-port metrics, signal diagnostics, heatmaps, and noise or interference measurements.
• 5G core data: Control-plane and inter-Network Function (NF) signaling data.
• IETF RFCs: The training corpus draws on 3GPP, ETSI, ITU, GSMA, CAMARA, O-RAN, and TM Forum material; it does not include a dedicated corpus of IETF RFCs (e.g., routing, MPLS, BGP, or other protocol RFCs).
• Vendor CLI and network operating system (NOS) documentation: Command references and configuration guides for vendor platforms such as Cisco IOS-XR and other Cisco NOS variants, Juniper JUNOS, DNOS, Nokia BNG platforms, Arista EOS, and similar equipment are not part of the training mixture.
• Operator-specific network design and configuration artifacts: Internal network design documents, control-plane design references, customer/equipment configuration records, and similar operator-proprietary engineering artifacts are not included.
• OSS/BSS and incident/change-management workflow tooling: Fault- and event-management consoles, IT service-management ticketing, and change-approval systems (e.g., alarm dashboards, trouble-ticket lifecycles, and MOP approval and closure records). Standards familiarity with TM Forum process frameworks does not imply validated ability to execute an incident through these operational systems.

Standards knowledge should not be interpreted as experience with live operator telemetry, private network records, vendor-specific CLI/NOS behavior, or these excluded operational datasets. Applications involving these data types require separate task-specific evaluation, grounding, and validation.

Operational Procedure (MOP) Validation Not Performed

The model has not been validated against Methods of Procedure (MOPs) — step-by-step operational runbooks (e.g., maintenance activities, configuration changes, turn-ups, failovers, troubleshooting sequences) — executed or emulated against live or digital-twin network device instances. No benchmarking has been performed to confirm that the model's step-by-step procedural reasoning (command sequencing, expected outputs, checkpoints, rollback steps) is correct or safe when applied to real or emulated network operating system instances. Standards and configuration-assistance knowledge demonstrated elsewhere in this model card should not be interpreted as validated readiness for autonomous or human-in-the-loop execution of operational procedures. Any such use requires dedicated MOP-based evaluation (e.g., against emulated NOS instances in a digital twin environment) before deployment.

Autonomous Operational Task Completion Not Benchmarked

The model has not been evaluated for autonomous, closed-loop completion of operational telecom tasks whose success is measured by the resulting state of a live or emulated network, rather than by the text of a response. Task families of this kind — extracting quantitative SLA/QoS parameters from a natural-language service request; bringing up and configuring a private 5G standalone core and RAN to meet measured latency and throughput targets; diagnosing and repairing a live routing fault through an alarm-to-closure workflow; and standing up a network defense that must withstand adversarial traffic — exercise capabilities the current training mixture targets but has not validated. Demonstrated standards knowledge and configuration assistance should not be read as readiness for autonomous completion of these tasks, which requires task-specific evaluation with outcome-based scoring before any non-supervised use.

Network Security Operations Not Validated

The model has not been trained or evaluated for defensive network-security operations — traffic and anomaly analysis, firewall/packet-filter or eBPF program construction, DDoS detection and mitigation, or deployment of long-running protective processes on network appliances. Security-operations use requires dedicated evaluation and human oversight.

General Reliability and Deployment Limitations

• This is the QLoRA variant: it was trained by adapting a 4-bit (NF4) frozen base, which is faster and cheaper but can yield lower quality than full-parameter bf16 fine-tuning. Where maximum quality matters, compare against the full-parameter OTel-2.0-LLM-31B-IT.
• OTel 2.0 is domain-specific to telecommunications and should not be treated as a general-purpose model for unrelated fields.
• Telecom standards evolve over time; answers should be checked against the relevant document version and release.
• RAG quality depends on document ingestion, chunking, retrieval, reranking, prompt design, and source freshness.
• Direct QnA behavior should be evaluated separately from RAG behavior; strong performance in one setting does not imply strong performance in the other.
• For high-impact operational use, deploy the model with retrieval, source attribution, validation checks, and human review appropriate to the system risk.

Future Work

The following work is planned and should not be interpreted as a capability claim for the current checkpoint:

• Release the exact OTel 2.0 training implementation and reproducible configuration.
• Release a comprehensive OTel 2.0 evaluation through MLPeFT in collaboration with MLCommons.
• Expand independent evaluation across direct question answering, RAG, abstention, instruction following, safety, and operational telecom tasks.
• Address the documented gaps in operational data, telecom-specific tool use, multilingual support, and telecom-specific multimodal evaluation.
• Evaluate Methods of Procedure, digital-twin workflows, closed-loop network tasks, and security operations before making operational-readiness claims.
• Work with the AWS and Tenstorrent teams to add and verify model training and inference support on their hardware platforms.
• Compare compatible training stacks for GRPO and multi-node training under a common benchmarking methodology.
• Benchmark AMD Ryzen inference across multiple models, inference stacks, and quantization formats.

Related Models

Language Models

• OTel LLM 2.0 Collection
• OTel LLM 1.0 Collection

Embedding Models

• OTel Embedding Collection

Reranker Models

• OTel Reranker Collection

Resources

• Project Page
• OTel code
• OTel Selected Coverage
• OTel Paper
• OTel 1.0 Coverage
• OTel 2.0 Coverage
• OTel Docker

Contributors and Organizations

Contributors to the OTel 2.0 release and supporting infrastructure include:

Organizations involved in the OTel 2.0 release and supporting infrastructure include AT&T, MLCommons, Microsoft, AMD, Dell, GSMA, Red Hat, and Pleias.

Citation

@inproceedings{tavakkoli2026otel2,
  title  = {OTel 2.0: Open Telco AI Datasets, Benchmarks, and Models},
  author = {Tavakkoli, Farbod and others},
  booktitle = {TBD},
  year   = {2026},
  url    = {https://huggingface.co/farbodtavakkoli}
}

Contact

If you have any technical questions, please feel free to reach out to mark.austin@att.com or farbod.tavakkoli@att.com or farbodtavakoli@gmail.com