Qwen3.5-35B-A3B — Telco Track A LoRA Adapter
A LoRA adapter distilled from a DeepSeek-V3-5 teacher on 5G NR drive-test troubleshooting trajectories (Zindi Telco Troubleshooting Agentic Challenge — Track A).
Model Details
| Property | Value |
|---|---|
| Base model | unsloth/Qwen3.5-35B-A3B |
| Architecture | Qwen3.5 MoE — 35B total params, ~3B active |
| Adapter type | LoRA (PEFT) |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| Target modules | q_proj, k_proj, v_proj, o_proj |
| Training precision | BF16 |
| Training method | Behavioral cloning / SFT distillation |
| Teacher model | DeepSeek-V3-5 (deepseek/deepseek-v3-5 via OpenRouter) |
| Task | 5G NR drive-test fault diagnosis (multi-choice) |
| Adapter size | ~33 MB |
Training Details
The adapter was trained with Unsloth + TRL SFTTrainer:
- Loss masking: assistant turns only (
train_on_responses_only) - Distillation: teacher (DeepSeek-V3-5) generated full agentic trajectories (tool calls + reasoning chain + final answer). Only trajectories with IoU ≥ 0.5 against ground truth were kept.
- Context window: 4096 tokens
- Effective batch: 8 (bs=1, grad_accum=8)
- Epochs: 3
- Learning rate: 2e-4 (cosine schedule, 5% warmup)
- Optimizer: AdamW 8-bit
- Hardware: 2× NVIDIA A40
Task Description
Given a 5G NR drive-test scenario (RF measurements, UE logs, network counters), the model uses agentic tool calls to investigate root causes and selects the correct fault categories from a multiple-choice list. Evaluation metric: IoU between predicted and gold answer sets.
How to Load
from unsloth import FastModel
from peft import PeftModel
model, tokenizer = FastModel.from_pretrained(
model_name="unsloth/Qwen3.5-35B-A3B",
max_seq_length=4096,
load_in_4bit=False,
load_in_16bit=True,
)
model = PeftModel.from_pretrained(model, "andreribeiro87/Qwen3.5-35B-A3B-telco-tracka-lora")
model.eval()
GGUF Versions
Quantized GGUF files (Q4_K_M and Q5_K_M) are available in the companion repo: andreribeiro87/Qwen3.5-35B-A3B-telco-tracka-GGUF
Framework Versions
- PEFT 0.19.1
- Unsloth (latest at training time)
- TRL SFTTrainer
- transformers ≥ 4.47