FogGen: edge-cloud router for multiple-choice questions
FogGen is Qwen3-0.6B fine-tuned to write a confidence value before its answer. If the confidence is at least a threshold tau, the edge answer is used. Otherwise the query goes to a cloud model (Qwen3-30B-A3B-Instruct-2507). This checkpoint is round 14 (R14) of a 14-round training chain over seven domains: finance, science, coding, law, math, Kazakh culture and medicine. In each round the current model answers each new training question 8 times, and the number of correct answers sets the confidence label it is trained on.
Data, per-round training files and per-question results: issai/foggen-data
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("issai/foggen", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("issai/foggen")
SYSTEM = """You are a self-aware multiple-choice assistant.
Rules:
- Do not output <think> tags.
- First, assess your confidence in solving this question.
- Then give your answer.
- Output format:
Confidence: <0.0|0.25|0.5|0.75|1.0>
Final answer: <OPTION_LETTER>"""
question = """A firm reports $400M in total liabilities and $600M in shareholders' equity.
What is the firm's debt-to-equity ratio?
A. 0.67
B. 1.00
C. 1.50
D. 2.00"""
messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": question}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True,
enable_thinking=False).to(model.device)
outputs = model.generate(inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
import re
def route_query(model_output, tau=0.5):
"""Returns (action, confidence, answer). An unparsed confidence is routed to the cloud."""
conf = re.search(r"Confidence\s*:\s*([\d.]+)", model_output)
ans = re.search(r"Final\s+answer\s*:\s*([A-D])", model_output)
if not conf:
return "route_to_cloud", None, None
c = float(conf.group(1))
return ("keep_local" if c >= tau else "route_to_cloud"), c, (ans.group(1) if ans else None)
Results
R14 at tau = 0.5 on the seven evaluation sets (16,217 questions). System accuracy counts the edge answer on kept queries and the cloud answer on routed queries. Random routing sends the same share of queries to the cloud at random. Scores use the corrected answer keys of issai/foggen-data, which leave out 1,582 items whose key could not be fixed.
| Domain | Questions | Cloud only | Edge only | Random routing | FogGen | Routed to cloud |
|---|---|---|---|---|---|---|
| Finance | 4,425 | 92.3% | 62.3% | 68.7% | 73.9% | 21.5% |
| Science | 1,742 | 93.6% | 61.4% | 67.9% | 72.2% | 20.1% |
| Coding | 1,762 | 93.5% | 64.8% | 70.1% | 75.0% | 18.6% |
| Law | 1,846 | 87.7% | 57.1% | 63.2% | 67.0% | 19.8% |
| Math | 1,703 | 95.5% | 46.6% | 69.5% | 77.0% | 46.8% |
| Kazakh culture | 1,317 | 96.2% | 90.9% | 90.9% | 91.5% | 1.1% |
| Medical | 1,840 | 88.9% | 57.4% | 63.8% | 68.5% | 20.2% |
| Mean | 92.5% | 62.9% | 70.6% | 75.0% | 21.2% |
FogGen is 4.4 points above random routing at R14 and 5.6 points on average over rounds 1 to 14. The mean expected calibration error is 8.3 at R14. Against the stored keys of the first release, R14 reaches 65.0% against 63.0% for random routing. evaluate.py in issai/foggen-data reproduces every number in this table.
Earlier versions of this card reported 67.8% system accuracy and a lift of 4.6 points. Those numbers credited routed queries with the cloud's average accuracy over the whole set instead of its answers to the routed queries, and they used the stored keys.
Training
| Base model | Qwen/Qwen3-0.6B |
| Cloud model and data generator | Qwen3-30B-A3B-Instruct-2507 |
| Rounds | R0 (1,800-row calibration buffer) and R1 to R14, one domain per round |
| Rows per round | 1,800: up to 300 new-domain rows, 60 replay rows per earlier domain, R0 buffer rows |
| Confidence label | correct answers in 8 samples at temperature 0.7, mapped to 0, 0.25, 0.5, 0.75 or 1 |
| Fine-tuning | LoRA rank 16, alpha 32, all linear layers, learning rate 5e-5, 2 epochs per round, bf16 |
Related checkpoints
issai/foggen-qwen3-1.7b, issai/foggen-gemma3-1b and issai/foggen-gemma3-270m run the same chain with other edge models. issai/foggen-r15-oe adds one round of open-ended tasks on top of this checkpoint.