Gold-supervised financial task adaptation
A Qwen3-0.6B model adapted for financial numerical reasoning and table-grounded question answering using gold-derived instruction-response supervision.
The strongest-performing student variant in this notebook's limited exploratory comparison. The experiment demonstrates a direct gold-supervision approach to adapting a compact general-purpose model for financial reasoning tasks.
The model was trained with parameter-efficient QLoRA supervised fine-tuning. Training targets were derived from FinQA and TAT-QA training annotations, including gold answers and arithmetic derivations, together with deterministically generated synthetic financial examples. No teacher-generated responses or teacher logits were used as supervision in this experiment.
The adaptation experiments used:
The published checkpoint contains the merged model weights, not an adapter-only repository.
| Component | Model |
|---|---|
| Student base | Qwen/Qwen3-0.6B |
| Teacher | Qwen/Qwen3-8B |
| Student revision | c1899de289a04d12100db370d81485cdf75e47ca |
| Teacher revision | b968826d9c46dd6066d109eabc6255188de91218 |
Hugging Face repository: https://huggingface.co/learner-subodh/qwen-finance-sft-0.6b
The model is intended for research and experimentation involving:
The model is designed to operate on supplied context rather than to retrieve current financial information independently.
The experiments use a structured response convention:
Calculation: <arithmetic expression = value, or n/a>
Answer: <final answer>
Scale: <thousand / million / billion / percent / none>
For insufficient context, the expected response is:
Answer: NOT_ANSWERABLE
Training data was prepared from:
For teacher-distillation experiments, teacher-generated responses were filtered using deterministic verification against training annotations.
Official validation and test examples were not intentionally used as training targets.
The experiments include a comparative evaluation against the original Qwen3-0.6B student and Qwen3-8B teacher.
The evaluation protocol used sampled FinQA and TAT-QA examples, identical prompts and greedy decoding across the compared models.
Important evaluation qualifications:
Consequently, this evaluation is exploratory and should not be interpreted as a full benchmark certification or a production performance guarantee.
This model is not validated for:
The model may produce incorrect calculations, misinterpret units, miss relevant table entries, or return an unsupported answer. Outputs should be checked against the original financial source and independently verified.
The experiments use a single training seed. Results from the limited evaluation sample do not establish generalization across financial domains, document formats or organizations.
Student base model:
https://huggingface.co/Qwen/Qwen3-0.6B
Teacher model:
https://huggingface.co/Qwen/Qwen3-8B
The student and teacher model revisions are recorded in the experiment configuration. Review upstream model and dataset licensing terms before redistribution or commercial use.
This project is an educational and experimental exercise. It was developed to explore small-language-model adaptation, supervised fine-tuning, response-based knowledge distillation and multi-stage training workflows. It is not a production-ready, financially validated or independently certified system.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "learner-subodh/qwen-finance-sft-0.6b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto"
)
messages = [
{
"role": "system",
"content": (
"You are a careful financial analyst. "
"Use only the supplied context. "
"Never invent values, periods, currencies or units."
)
},
{
"role": "user",
"content": (
"Context:\\n"
"Fiscal year | 2021 | 2022\\n"
"Revenue | 100 | 120\\n\\n"
"Question: What was the percentage change in revenue?\\n"
"Respond in Calculation / Answer / Scale format."
)
}
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer(
prompt,
return_tensors="pt"
).to(model.device)
with torch.inference_mode():
output = model.generate(
**inputs,
max_new_tokens=128,
do_sample=False
)
new_tokens = output[0][inputs["input_ids"].shape[1]:]
print(
tokenizer.decode(
new_tokens,
skip_special_tokens=True
)
)
If you use this model or its training methodology in an educational or research project, cite the upstream Qwen3 model, FinQA, TAT-QA and the relevant parameter-efficient fine-tuning work. Also cite this repository and its accompanying experiment notebook.
Disclaimer: This is an educational exercise, not a production-ready financial reasoning system.
Qwen/Qwen3-0.6B