UDGAM LABS · Tool-calling model, MLX 4-bit · 384M parameters

Udgam-384M-Chat-MLX-4bit

This repo holds the MLX 4-bit version of Udgam-384M-Chat for Apple silicon: pip install mlx-lm, then mlx_lm.generate --model udgamlabs/Udgam-384M-Chat-MLX-4bit --prompt "..." (or mlx_lm.server for an OpenAI-compatible API). Converted with mlx-lm 0.32 (4-bit, group size 64, 4.5 bits per weight). In our checks 4-bit MLX greedy output matched the full-precision output on 4 of 12 test prompts (fp16 MLX: 11 of 12); prefer the fp16 MLX or the Q8_0 GGUF when exact tool calls matter. The rest of this page is the main model card.

A 384M-parameter English model for on-device tool calling, trained from scratch by Udgam Labs. It is a decoder-only transformer compatible with the Gemma 4 architecture (Gemma4ForCausalLM), so it runs unmodified in Ollama, LM Studio, llama.cpp, MLX and transformers. Context: 4,096 tokens. The 4-bit GGUF is 238 MB. Built for four jobs inside an app:

1. App actions: turn a request into a function call. "wake me at 6:45, label it gym" → set_alarm{time:"06:45", label:"gym"}.
2. Document → JSON: fill a given JSON schema from a receipt, email, form or OCR text.
3. Natural language → shell: propose one command (as a run_shell_command call with requires_confirmation, or as a plain command block) for a person to review.
4. Relevance: notice when no offered tool fits (decline), when a required detail is missing (ask), or when a request should be refused.

What it is not: a knowledgeable general assistant (at 384M parameters it makes factual mistakes), or a safety-hardened open chatbot. Use it where your app decides which tools exist and confirms risky actions.

Model family: Udgam-384M-Base (2,048 context) · Udgam-384M-Base-4K · Udgam-384M-Chat · GGUF · MLX · MLX 4-bit · Technical report

Quick start

Ollama (recommended)

Download a GGUF from the GGUF repo and create a model with its Modelfile (it uses Ollama's native Gemma 4 renderer and parser, so tool calls and the optional thinking mode work as trained):

FROM ./Udgam-384M-Chat-Q8_0.gguf
TEMPLATE {{ .Prompt }}
RENDERER gemma4
PARSER gemma4
PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1
PARAMETER repeat_last_n 64
PARAMETER num_predict 1024
PARAMETER num_ctx 4096

ollama create udgam -f Modelfile
ollama run udgam "Explain what a hash map is in two sentences."
curl http://localhost:11434/api/chat -d '{
  "model": "udgam", "stream": false, "think": false,
  "messages": [
    {"role": "system", "content": "Current date-time: Tuesday, 2027-08-17 09:12 (Europe/London, UTC+01:00)\nDevice: Android phone"},
    {"role": "user", "content": "Wake me up at 6:45 tomorrow, label it gym."}],
  "tools": [{"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm at a local time.",
     "parameters": {"type": "object", "properties": {"time": {"type": "string", "description": "HH:MM 24h"},
                    "label": {"type": "string", "description": "Optional label"}}, "required": ["time"]}}}]}'
# -> "tool_calls": [{"function": {"name": "set_alarm", "arguments": {"label": "Gym", "time": "06:45"}}}]

Thinking is on by default in Ollama for Gemma 4 models; send "think": false (or /set nothink) to turn it off. For this model thinking is optional and usually not better (see the tables).

LM Studio

1. Search for udgamlabs/Udgam-384M-Chat-GGUF (or drop the .gguf into LM Studio's models folder) and pick Q8_0 (default) or Q4_K_M (phones / low memory).
2. The GGUF carries the Gemma 4 chat template, so the prompt format is set automatically. Set Context Length 4096, temperature 0.1, top-p 0.9, top-k 40, repeat penalty 1.1.
3. For tool use, call LM Studio's OpenAI-compatible server (/v1/chat/completions with tools). If your version does not parse Gemma 4 tool calls into tool_calls, read them from the text: <|tool_call>call:NAME{key:<|"|>value<|"|>,...}<tool_call|>.
4. Using this MLX version in LM Studio: before loading, open the model's load settings (the gear icon, or turn on "Manually choose model load parameters") and set Max Concurrent Predictions to 1. With the default (parallel) setting, LM Studio's MLX engine (tested with mlx-llm 1.13.1) fails on text-only Gemma 4 models with KeyError: 'text_config'. The GGUF version needs no change.

llama.cpp

llama-server -m Udgam-384M-Chat-Q8_0.gguf -c 4096 --jinja                  # OpenAI-compatible API, parsed tool_calls
llama-server -m Udgam-384M-Chat-Q8_0.gguf -c 4096 --jinja --reasoning off  # same, thinking off
llama-cli    -m Udgam-384M-Chat-Q8_0.gguf -c 4096 -cnv                     # interactive chat

With the set_alarm tool above, /v1/chat/completions returns tool_calls: [{"name": "set_alarm", "arguments": "{\"label\":\"Gym\",\"time\":\"06:45\"}"}] (tested with llama.cpp build d7a695e).

MLX (Apple silicon)

pip install mlx-lm
mlx_lm.generate --model udgamlabs/Udgam-384M-Chat-MLX-4bit --prompt "Give me three tips for sleeping better." --max-tokens 200

In LM Studio, load this model with Max Concurrent Predictions = 1 (see the LM Studio section above).

transformers

# tested with transformers 5.18
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("udgamlabs/Udgam-384M-Chat")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Udgam-384M-Chat")
tools = [{"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm at a local time.",
          "parameters": {"type": "object", "properties": {"time": {"type": "string", "description": "HH:MM 24h"},
                         "label": {"type": "string"}}, "required": ["time"]}}}]
msgs = [{"role": "user", "content": "Wake me up at 6:45, label it gym."}]
inputs = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True, return_dict=True, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)   # thinking off unless enable_thinking=True
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False))
# <|tool_call>call:set_alarm{label:<|"|>Gym<|"|>,time:<|"|>06:45<|"|>}<tool_call|><|tool_response>

Use fp32 (or llama.cpp) if you need bit-exact greedy results; bf16 generation in transformers depends on batch shape.

Prompt format

The standard Gemma 4 chat format (the template is identical to the official one). A tool turn:

<bos><|turn>system
Current date-time: Tuesday, 2027-08-17 09:12 (Europe/London, UTC+01:00)
Device: Android phone<|tool>declaration:set_alarm{description:<|"|>Set an alarm at a local time.<|"|>,parameters:{properties:{label:{description:<|"|>Optional label<|"|>,type:<|"|>STRING<|"|>},time:{description:<|"|>HH:MM 24h<|"|>,type:<|"|>STRING<|"|>}},required:[<|"|>time<|"|>],type:<|"|>OBJECT<|"|>}}<tool|><turn|>
<|turn>user
Wake me up at 6:45 tomorrow, label it gym.<turn|>
<|turn>model
<|tool_call>call:set_alarm{label:<|"|>gym<|"|>,time:<|"|>06:45<|"|>}<tool_call|><|tool_response>

The model stops at <|tool_response>; your app runs the tool, sends the result back as a tool message, and the model writes the final reply. With thinking on, <|think|> is added after <|turn>system\n and the model may write <|channel>thought\n…<channel|> first.

Tips: put the current date-time, time zone and device in the system message (as above) so relative dates resolve; give one JSON schema (or field list) for document → JSON; offer only the tools the user may use right now.

Example outputs (Ollama, Q8_0, thinking off): "Text Priya that I'll be 10 minutes late." → send_message{contact:"Priya", text:"I'll be 10 minutes late"}; "Send a message to Rahul." → "What should the message to Rahul say?"; "Book me a table for two at 8pm." (only alarm / message / weather tools offered) → "I can't make restaurant bookings here. Want me to set an alarm for 8 pm so you can ring them?"; a receipt → {"vendor": "Corner Cafe", "total": 8.9, "date": "2027-08-14"}. A typical mistake: "What's the boiling point of water in Fahrenheit?" → "about 37 degrees Celsius, roughly 100 degrees Fahrenheit" (wrong).

Recommended quantization and sampling

Quantization: use Q8_0 by default (lossless within noise) and Q4_K_M on phones / low-memory devices (small loss, see below). Avoid Q4_0 unless your runtime lacks K-quants.

Why: temperature is what costs tool accuracy (BFCL simple in Ollama: 60% greedy, 59% at T 0.1, 52% at 0.5, 46% at 0.7), while a mild repeat penalty removes repetition loops without hurting tool calls. In transformers, generation_config.json sets T 0.1 / top-p 0.9 / top-k 40 (no repetition_penalty: transformers applies it to the prompt too, which would penalise tool names). Use greedy decoding for reproducible tool calls.

Evaluation

Automatic checkers, greedy decoding with llama.cpp on a Mac. The held-out task evals are our own test sets (274 items) built on tools, document types and shell tasks that were excluded from all training data, plus a held-out date range. They follow this model's prompt conventions, so they favour it; BFCL v3, IFEval and XSTest are neutral public benchmarks.

Compared with other tiny open models (same harness, same prompts)

Each model used its own chat template and native tool-call format; outputs were scored by the same code.

Speed and memory (llama.cpp llama-bench, Q4_K_M, 512-token prompt, 128 generated tokens, Apple M6 Mac mini):

RAM = macOS peak memory footprint of the CPU run. Phones are slower; the relative order should be similar on CPU.

Where Udgam does well: deciding whether to act: declining when no offered tool fits (held-out relevance 75% vs 18–20% for both Granite models, which call a tool almost every time), asking for a missing detail (20–25% vs 0–5%), document → JSON against an unseen schema (schema-valid 56–60% vs 24–38%) and shell behaviour (86% vs 59–67%). It is also the fastest generator in the group on the Apple-silicon GPU (444 tokens/s at Q4_K_M).

Fairness notes:

• The held-out task sets use this model's prompt conventions (date/device system headers, normalised argument values such as ISO dates); other models lose some points to convention rather than ability. BFCL, IFEval and XSTest are neutral.
• On the neutral BFCL irrelevance test, Qwen3-0.6B (88%) and SmolLM2 (75%) beat Udgam (68%); Udgam's relevance advantage shows mainly on our own held-out relevance set.
• Granite and Qwen3 used their native templates with tools; SmolLM2-360M's template has no tool support, so it got the function-calling system prompt from its model card. It mostly copied the example call from that prompt, so its tool scores (≈0–5%) mean "not a tool model", not "bad model".
• Qwen3 ran with thinking off; Udgam with thinking off; Granite and SmolLM2 have no thinking mode. All greedy, Q8_0, same token limits, same scorer.
• XSTest uses one refusal-phrase classifier for every model (first 300 characters); on this model it agreed with an LLM judge within 3 points.
• Not compared: FunctionGemma-270M and Gemma 3 270M (gated licence terms on Hugging Face; not evaluated).

Quantization

Perplexity on held-out chat and tool-use text: f16 3.193, Q8_0 +0.07%, Q6_K +0.5%, Q4_K_M (imatrix) +4.2%, Q4_0 (imatrix) +8.3%.

Where it falls short

It is behind on exact tool calls and instruction following. Both Granite 4.0 350M models are far ahead on BFCL (simple 88–91% vs 60%, parallel 76–82% vs 46%), on held-out unseen-tool exact match (46% vs 31%) and on IFEval (47–53% vs 26%). Qwen3-0.6B (about 1.6 times the parameters) is also ahead on BFCL and IFEval. Shell commands are rarely correct when executed for any model here (0–8%).

On the neutral BFCL irrelevance test, Qwen3-0.6B and SmolLM2 also beat it (see the fairness notes above). For safety results and the tasks it is not meant for, see Safety and Intended use and limitations.

Safety

XSTest (200 clearly unsafe + 250 safe-but-alarming prompts), greedy, thinking off: the model declines 54% of clearly harmful prompts and over-refuses 15% of safe ones (refusal-phrase classifier, the same one used for every model compared; Q8_0 and Q4_K_M identical; an LLM judge gave 56.5% and 14.4%). It rarely declines XSTest's discriminatory-claim, privacy and historical-atrocity prompts, and it over-refuses idioms and homonyms ("kill time", "shoot a photo"). This is below what we would want for an open chatbot; treat the model as not safety-hardened.

• For app actions: rely on your app's allowlist of tools, argument validation and user confirmation for anything with consequences (send, pay, delete, share, run); do not rely on the model's judgement.
• For shell commands: always show the command and require confirmation; never auto-execute.
• For user-facing open chat: put a moderation model or policy layer in front.

Intended use and limitations

Good fits: on-device assistants that map requests to a fixed set of app functions; form filling and receipt / email → JSON with a given schema; suggesting shell commands for a person to review; routing (call / ask / decline).

Use with care / not intended for:

• Unsupervised actions. It picks a wrong tool or argument value often enough (exact match on unseen tools is about one in three) that any consequential action needs app-side allowlists, validation and confirmation.
• Shell commands: syntax is usually valid, but on unfamiliar tasks only a small fraction produce the right output, and it does not reliably refuse destructive requests.
• Unsupervised open-ended chat or advice: fluent but shallow and often factually wrong (capitals, arithmetic, unit conversions). Not for medical, legal or financial use.
• Safety: see above (54% of harmful prompts declined, 15% over-refusal).
• Context: 4,096 tokens. English only.
• It prefers its own argument conventions (ISO dates, normalised values) and sometimes guesses a value instead of asking.

Training

Pretrained from scratch on roughly 8B tokens of licensed public and synthetic text (base: Udgam-384M-Base-4K), then post-trained with supervised fine-tuning for chat and tool use. See the technical report for an overview.

Licence

Apache-2.0 (weights and tokenizer); see LICENSE. The chat template is the official Gemma 4 template (Apache-2.0).

Citation

@misc{udgamlabs2026udgam384m,
  title        = {Udgam-384M: a small on-device tool-calling model trained from scratch},
  author       = {{Udgam Labs}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/udgamlabs/Udgam-384M-Chat}},
  note         = {Technical report: \url{https://udgamlabs.com/reports/udgam-384m}}
}

Contact

Udgam Labs. Please use the Community (discussions) tab of the model page for questions, issues and feedback.

---

About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com