GPT-Flux v1 — LoRA Adapter

QLoRA adapter (r=64, α=128, 4-bit NF4) for GPT-Flux v1, the Chinese geoscience / flux-tower Q&A model. Trained 3 epochs on 6,175 QA pairs from 1,609 papers; final train loss 1.76 on a single A100 (~24 min).

For most users, use the merged model instead: LonghaoWang/flux-gpt-qwen3-14b-merged — no adapter code needed.

Usage (4-bit, fits a single RTX 4090 / free Colab T4)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16)
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B",
    quantization_config=bnb, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(base, "LonghaoWang/flux-gpt-qwen3-14b")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-14b", trust_remote_code=True)

Links

• 🤗 Collection: GPT-Flux
• 💻 Code, docs & training recipe: GISWLH/GPT-Flux
• 📦 Training data: LonghaoWang/flux-sft