Qwen3.8-27B fine-tuned on synthetic Charcot Gym episodes for healthcare admin work: claims, clinical coding, and document → FHIR R5. On private tasks it has never seen, the mean score rises from 0.505 to 0.727.
| Field | Value |
|---|---|
| Base model | Qwen/Qwen3.8-27B |
| Training data | charcot-ai/gym-sft |
| Benchmark | charcot-ai/gym · Charcot-AI/gym |
| Input / output | Text in, text and tool calls out (vision tower unchanged) |
| License | Apache-2.0 |
Serve the model with an OpenAI-compatible API:
vllm serve charcot-ai/charcot-27b --dtype bfloat16 --max-model-len 131072
Send a request:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="charcot-ai/charcot-27b",
messages=[{"role": "user", "content": "Which ICD-10-CM code fits type 2 diabetes without complications?"}],
)
print(response.choices[0].message.content)
Evaluate it on the public Charcot Gym split:
pip install "git+https://github.com/Charcot-AI/gym.git@v1.1.0"
export OPENAI_API_KEY=EMPTY
charcot-gym eval \
--model charcot-ai/charcot-27b \
--base-url http://localhost:8000/v1 \
--api-key-env OPENAI_API_KEY \
--split public --text-only --attempts 3 \
--out results/charcot-27b
The run writes summary.json with overall and per-family scores.
Mean score from 0 to 1, 3 attempts per task.
| Evaluation | Tasks | Qwen3.8-27B | Charcot 27B |
|---|---|---|---|
| Private split | 162 | 0.51 | 0.73 |
| Public held-out | 144 | 0.37 | 0.54 |
Private split by family:
| Family | Qwen3.8-27B | Charcot 27B | Change |
|---|---|---|---|
| Claims | 0.37 | 0.71 | ↑ |
| Clinical coding | 0.42 | 0.69 | ↑ |
| Document → FHIR | 0.73 | 0.78 | ≈ |
↑ gain, ↓ drop, ≈ no significant difference (95% confidence interval of the difference).
| Field | Value |
|---|---|
| Data | 801 synthetic text episodes: 153 public-split tasks and 648 generated with the public generators (8 published seeds) |
| Method | LoRA SFT (rank 32) on the response only, merged into FP32 weights |
| Private data | None: no private task or seed was used in training |
@misc{charcot2026charcot27b,
title = {Charcot 27B},
author = {{Charcot AI}},
year = {2026},
url = {https://huggingface.co/charcot-ai/charcot-27b}
}
Apache-2.0. The base model remains subject to its own license.
model-00004-of-00018.safetensors 8.0 GBmodel-00016-of-00018.safetensors 8.0 GBmodel-00006-of-00018.safetensors 8.0 GBmodel-00008-of-00018.safetensors 8.0 GBmodel-00010-of-00018.safetensors 8.0 GBmodel-00012-of-00018.safetensors 8.0 GBmodel-00014-of-00018.safetensors 8.0 GBmodel-00001-of-00018.safetensors 7.9 GBmodel-00018-of-00018.safetensors 6.8 GBmodel-00002-of-00018.safetensors 6.1 GBmodel-00003-of-00018.safetensors 5.1 GBmodel-00007-of-00018.safetensors 4.2 GBmodel-00009-of-00018.safetensors 4.2 GBmodel-00011-of-00018.safetensors 4.2 GBmodel-00013-of-00018.safetensors 4.2 GBmodel-00015-of-00018.safetensors 4.2 GBmodel-00017-of-00018.safetensors 4.2 GBmodel-00005-of-00018.safetensors 4.2 GBQwen/Qwen3.8-27B