Gemma 700M IT · Racore

A compact Gemma 3 1B IT derivative with mixed-precision weight storage and an explicit NumPy inference engine.

racore/gemma-700m-it-racore packages a derivative of Google Gemma 3 1B IT using 4-bit MLP weights, 8-bit attention weights, and 8-bit embeddings. The repository includes a custom NumPy runtime, tokenizer wrapper, mathematical descriptions, and experimental benchmark artifacts.

Naming: “700M” refers approximately to the stored package size. It does not indicate a model with 700 million total parameters. The declared tensor payload is 729,892,352 bytes, approximately 729.9 MB / 696.1 MiB, excluding tokenizer, source code, and other repository files.

Model overview

English and Arabic are listed as intended usage languages. This release does not establish comparable quality across languages through a published multilingual evaluation.

How it works

Mixed-precision weight representation

Quantized weights are reconstructed using stored integer values and scales:

$$ \widehat{W}{ij}=s{i,g(j)}q_{ij}. $$

For grouped MLP quantization, $g(j)=\lfloor j/32\rfloor$. The package retains the underlying transformer structure while reducing the storage precision of large tensors.

Quantization introduces approximation error. Expressing the forward pass as equations does not by itself make the model lossless or fully interpretable.

Explicit NumPy forward pass

The custom engine implements embedding lookup, RMS normalization, rotary position embeddings, attention, gated MLP blocks, residual connections, and autoregressive generation with a KV cache.

The repository also contains factorization and polynomial-surrogate experiments. Their individual benchmark results should be distinguished from the released q4_mix package and from complete language-model generation.

Quick start: CPU

Use Python 3.10 or later. A virtual environment is recommended.

python -m pip install numpy tokenizers huggingface_hub

Download the repository:

from huggingface_hub import snapshot_download

model_dir = snapshot_download(
    repo_id="racore/gemma-700m-it-racore",
    local_dir="gemma-700m-it-racore",
)
print(model_dir)

The snapshot includes executable Python source. Review the downloaded code before running it.

Python example

Run this example from the directory containing gemma-700m-it-racore:

from pathlib import Path
import sys
import numpy as np

root = Path("gemma-700m-it-racore").resolve()
sys.path.insert(0, str(root))

from engine import SymbolicModel
from tokenizer import GemmaTokenizer

tokenizer = GemmaTokenizer(str(root))
model = SymbolicModel.from_package(str(root / "equations" / "q4_mix"))

# The lightweight tokenizer wrapper does not apply a chat template for you.
prompt = (
    "<start_of_turn>user\n"
    "Explain machine learning in two short sentences."
    "<end_of_turn>\n"
    "<start_of_turn>model\n"
)
input_ids = np.asarray(tokenizer.encode(prompt), dtype=np.int64)

output_ids = model.generate(
    input_ids,
    max_new_tokens=64,
    temperature=0.0,
    eos_ids=(1, 106),
)

# generate() returns the prompt and generated continuation.
new_ids = output_ids[len(input_ids):]
print(tokenizer.decode(new_ids, skip_special_tokens=True))

For Arabic, replace the user text with a prompt such as:

question = "اشرح التعلم الآلي في جملتين بسيطتين."

Wrap it in the same turn markers shown above. Generated answers can contain mistakes.

CLI example

cd gemma-700m-it-racore
python generate.py --device cpu --prompt "The capital of Egypt is" --max-new-tokens 24 --temperature 0

Use the ordinary elapsed-time output to assess actual throughput. The current CLI additionally prints speculative acceleration figures calculated from fixed multipliers; these figures are estimates rather than timings of a speculative decoding execution.

Compatibility and memory

• The CPU path uses NumPy for inference and tokenizers for tokenization. huggingface_hub is needed for downloading.
• The package has a custom format. Standard transformers.AutoModelForCausalLM.from_pretrained() loading is not configured for this release.
• The current CUDA CLI branch attempts to load a conventional Transformers checkpoint and includes a developer-local fallback path. It should not be treated as a portable GPU loader for the packaged q4_mix tensors.
• Disk size is not peak RAM usage. Dequantization, cached dense tensors, activations, logits, and the KV cache can increase memory substantially.
• Throughput depends on CPU, BLAS backend, thread count, prompt length, and generation length.

Storage and evaluation

Storage declared by the package manifest

The bits-per-layer-weight value excludes the separately reported embedding storage and is not a whole-repository average.

Benchmark interpretation

The repository contains the following research artifacts:

Evaluation status: this card does not claim measured perplexity, universal token equivalence, lossless compression, or verified end-to-end acceleration for the released package. Those require a reproducible evaluation of this exact artifact against the original checkpoint.

A useful evaluation should publish prompt sets, token counts, decoding settings, software versions, hardware, warm-up policy, wall-clock timings, peak memory, perplexity, and task-quality results. Exact-match observations should always specify the tested inputs and decoding settings.

Intended uses and limitations

Suitable uses include local inference experiments, mixed-precision storage research, educational inspection of transformer equations, and investigation of symbolic or factorized approximations.

The model can hallucinate, reproduce biases inherited from its base model, and lose quality after quantization. Mathematical visibility into the forward computation does not establish a semantic explanation for every generated answer. Long-context reliability and Arabic quality should be evaluated for the intended application.

Repository guide

License

The repository declares the Gemma license. Consult the Gemma Terms of Use, applicable usage requirements, and the base model card before using or redistributing this derivative. Compression does not remove the base model's applicable terms.

Author and citation

Loaii Abdalslam · Racore
Website: racore.xyz
Model: racore/gemma-700m-it-racore

@misc{abdalslam2026gemmaracore,
  author       = {Abdalslam, Loaii},
  title        = {Gemma 700M IT Racore},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/racore/gemma-700m-it-racore}
}