Qwen3.5-4B-3D-Coder

Early preview · General coding with a focus on 3D

An experiment exploring the feasibility of small language models for general-purpose coding and high-quality 3D scene generation.

This is an early preview of a 4B model I am still training, with a particular focus on creating interactive 3D scenes with Three.js. Initial hands-on results have been promising for coding in general, and especially for generating 3D scenes, so I decided to share the progress while development continues.

This is an independent fine-tune by CCORE, based on Qwen/Qwen3.5-4B. It is not an official Qwen release.

Optional vision support — October 30, 2026

The Q4_K_M and Q8_0 GGUF versions can also accept image inputs when loaded with the original Qwen3.5-4B vision encoder/projector. You can attach a reference image and ask the model to generate a Three.js scene from it.

Download the optional BF16 vision projector — approximately 644 MiB. Download it once; the same file is used with either GGUF quantization.

Image-guided Three.js example

With a reference image attached and the Three.js system prompt below, the user prompt was simply:

create this scene using voxels 3d

The reference image was supplied in community discussion #9; credit for the reference belongs to its original creator. The right-hand image is a screenshot of the generated scene. This is a qualitative example of image-guided generation; the output interprets the reference and does not reproduce every detail.

Load the GGUF with vision in llama.cpp

Download a model GGUF and the projector, then load both files explicitly:

llama-server \
  -m Qwen3.5-4B-3D-Coder-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.5-4B-BF16.gguf \
  --n-gpu-layers all --flash-attn on \
  --ctx-size 65536 --parallel 1

Open the server's WebUI, attach an image, and send your prompt. The setup was tested locally with llama.cpp b11889. For Q8_0, change the model filename to Qwen3.5-4B-3D-Coder-Q8_0.gguf; --cache-type-k q8_0 --cache-type-v q8_0 can reduce KV-cache memory use and was tested locally with vision at a configured 64K context. These were short tests, not full-context stress tests.

The projector uses additional memory when loaded, and image tokens share the context window with the prompt and generated code. Image support in other frontends depends on their multimodal implementation.

Projector provenance: this is the unmodified BF16 projector from lmstudio-community/Qwen3.5-4B-GGUF, based on Qwen/Qwen3.5-4B, released under Apache 2.0. It is an optional companion file for the existing GGUF weights.

Technical report — now available

The technical report for this training experiment is now available.

Read the technical report (PDF) · Download the PDF

Full FP16 weights — now available

The original FP16 weights are now available in this repository in Safetensors format, alongside the Q4_K_M and Q8_0 GGUF versions.

The release includes all four weight shards, the model configuration, tokenizer, generation configuration, chat template, and image/video processor settings. The FP16 weights were verified locally for loading and text generation with Transformers 5.20.2.

The FP16 generation defaults use temperature 0.6, top-k 20, top-p 0.95, and repetition penalty 1.0, with caching enabled.

Browse the FP16 files and GGUF versions

Training dataset — now available

The dataset used to train this model is now public: CCORE Three.js 3D Scenes.

It contains 4,146 scene examples (3,933 training / 213 validation), including prompts, full thinking traces, complete HTML/JavaScript responses, and complexity levels. Rendered reference images are available in the optional previews configuration. The dataset is released under the MIT License.

October 27 checkpoint update

Both Q4_K_M and Q8_0 now contain the updated October 27 checkpoint. If you downloaded an earlier version, please download the files again to get this update.

In my latest tests, one-shot success has improved substantially, along with overall scene quality. Some scenes can still require another attempt or a correction.

Generation parameters remain unchanged for both quantizations. Use the settings listed below. The earlier fix for the layer-related issue reported by some users is also included.

The comparison below shows the Rome scene across the October 25, 26, and 27 previews.

Release update — October 29, 2026

The full FP16 weights, training dataset, and technical report (PDF) are now available.

The full set of quantized versions is also planned.

Early examples

Selected screenshots of rendered Three.js scenes: stylized architecture, gardens, voxel environments, furniture, workshop interiors, and mechanical objects, built through HTML and JavaScript.

Click any image to view the original screenshot at full size.

Generation settings

Use the following settings for both the Q4_K_M and Q8 versions:

LM Studio: disable MTP

Important: If you're running this model in LM Studio, disable multi-token prediction (MTP) for the model to work properly. LM Studio enables this option by default.

System prompt for Three.js scenes

Use only the Three.js core ES module from
https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js.
Do not import addons or legacy examples/js paths;
implement any camera controls directly inside the HTML.

Example prompts

Use the system prompt above, then try one of these user prompts.

Elephant rehabilitation habitat

Please create one self-contained HTML file with Three.js from cdnjs showing
an elephant rehabilitation habitat.

A broad sheltered yard connects water, shade, and feeding areas through
accessible paths. Use the elephant's scale and movement clearance to guide
the layout around the structures.

Render it as a deliberate 3D voxel model built from visible cubic blocks,
with clear depth, readable silhouettes, and solid spatial relationships.

Dragonrider aerie on a sea stack

Build a self-contained HTML scene with Three.js from cdnjs featuring a
dragonrider aerie on a sea stack, with perches, a hatchery, and a windlass.

The aerie occupies a flat-topped sea stack, reached by a rope windlass from
the shore. Four projecting stone perches ring the central hatchery, a
vaulted cave with sand nests and a smoke hole. A timber landing deck holds
a hoist crane and harness racks, while switchback stairs connect the
perches to a water-level boathouse.

Render it as a deliberate 3D voxel model built from visible cubic blocks,
with clear depth, readable silhouettes, and solid spatial relationships.

Canal lock and weir complex

Please create one self-contained HTML file with Three.js from cdnjs showing
a canal lock and weir complex with a central island.

Animated air traffic control tower

Make a standalone HTML scene with Three.js from cdnjs depicting an animated
air traffic control tower with rotating radar and light signals.

A tapered control tower rises from a rectangular base to a cantilevered
glass cab with angled glazing panels and an external catwalk. A primary
surveillance radar on a roof pedestal turns continuously, and a rotating
beacon sits above the cab.

Known issues

The bugs observed so far have generally involved duplicate variable names, rather than major structural problems.