DroidSenior-7B v3
An Android and Kotlin coding model. Continues from DroidSenior v1/v2, whose training data included JetBrains/KExercises (CC-BY-NC-SA-4.0). Check those terms before commercial use.
System prompt
You are DroidSenior, a staff-level Android engineer. Write idiomatic, modern Kotlin (Jetpack Compose, Coroutines/Flow, Hilt, Room) following the official Android architecture guidance. Explain key design decisions and trade-offs briefly, never use deprecated APIs, avoid over-engineering, and include tests when they add value. Answer in the user's language.
Training
- SFT: 6026 examples, LoRA r=32, loss on assistant tokens only.
- DPO: 1208 compiler-verified pairs (the rejected answer failed to compile), beta=0.1.
- Questions from the evaluation split were removed from training data.
Evaluation
All models used the same system prompt and deterministic generation. Rates show 95% bootstrap intervals.
| model | Android compile rate | Kotlin HumanEval pass rate | legacy API rate |
|---|---|---|---|
| base | 2.6% (0%-6%) | 62.1% (55%-70%) | 40% |
| v1 | 5.1% (1%-10%) | 64.0% (57%-71%) | 3% |
| v2 | 3.8% (0%-9%) | 64.6% (57%-72%) | 4% |
| sft_r2 | 20.5% (12%-29%) | 67.7% (60%-75%) | 0% |
| dpo_r2 | 20.5% (12%-29%) | 67.7% (61%-75%) | 0% |
- Android compile rate: answers to 78 held-out questions, each asking for self-contained code, compiled with
:app:compileDebugKotlinin a fixed template project, without KSP. It measures compilation, not behavior. - Kotlin HumanEval: 161 tasks from JetBrains/Kotlin_HumanEval.
- Selected model:
dpo_r2. Android difference againstv1: +15.4% (95% CI +6.4% to +24.4%).
Q4_K_M GGUF: 4.68 GB, CPU speed 7.3 tokens/s on Colab.
Limitations
- Generated code can still be wrong. Review it before use.
- Recent APIs may be missing.
- These are self-run tests, not an official Android Bench result.