Hybrid Intelligence 0.5B GGUF

This repository provides GGUF quantizations for Merlin-Research/HybridIntelligence-0.5B.

Hybrid Intelligence 0.5B is built on the Falcon-H1 hybrid architecture, combining Mamba2 state-space blocks (SSM) with standard multi-head attention layers across 36 hidden layers. It couples causal language generation with neuromorphic uncertainty signals, optimized through Direct Preference Optimization (DPO) and biological neural network (BNN) calibration.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, followed by standard k-quant quantization.

Available Files and Quantizations

File	Quant Type	Size	Description / Recommendation
`HybridIntelligence-0.5B-BF16.gguf`	BF16	996.36 MB	Native bfloat16 base conversion. Highest fidelity reference weights.
`HybridIntelligence-0.5B-Q8_0.gguf`	Q8_0	530.54 MB	Near-lossless 8-bit quantization. Recommended for best quality evaluation.
`HybridIntelligence-0.5B-Q6_K.gguf`	Q6_K	410.21 MB	High quality retention with minimal degradation. Excellent accuracy/memory balance.
`HybridIntelligence-0.5B-Q5_K_M.gguf`	Q5_K_M	353.55 MB	Good balance between memory consumption and text generation fidelity.
`HybridIntelligence-0.5B-Q4_K_M.gguf`	Q4_K_M	300.22 MB	Fast, lightweight 4-bit quant. Recommended default for edge devices and mobile setups.
`HybridIntelligence-0.5B-Q3_K_M.gguf`	Q3_K_M	241.70 MB	Compact footprint when memory headroom is strictly limited.
`HybridIntelligence-0.5B-Q2_K.gguf`	Q2_K	191.60 MB	Maximum compression for ultra-constrained low-RAM environments.

Model Architecture Details

Base Architecture: FalconH1ForCausalLM (falcon_h1)
Layers: 36 (hybrid Mamba2 SSM + Multi-Head Attention)
Hidden Size: 1024
Attention Heads: 8 query heads / 2 KV heads, head dimension 64
Mamba Heads: 24 heads, state dimension 128, conv kernel 4
Intermediate Size: 2048
Context Length: 16,384 tokens
Vocabulary Size: 32,784
Native Dtype: bfloat16

Prompt Format and Usage

Falcon-H1 uses standard autoregressive completion format:

The key difference between biological and artificial neural networks is

Quickstart Guide

1. llama.cpp

Run generation with llama-cli:

llama-cli -m ./HybridIntelligence-0.5B-Q4_K_M.gguf \
    -p "The emergence of intelligence in complex adaptive systems can be described as" \
    -n 256 \
    --temp 0.7 \
    --top-p 0.9 \
    --repeat-penalty 1.1

Launch a local inference server with llama-server:

llama-server -m ./HybridIntelligence-0.5B-Q4_K_M.gguf \
    --port 8080 \
    -c 4096

2. Ollama

Create a Modelfile:

FROM ./HybridIntelligence-0.5B-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9

Build and run:

ollama create hybrid-intelligence -f Modelfile
ollama run hybrid-intelligence "Explain state-space models in simple terms."

3. LM Studio

Copy the .gguf file to your LM Studio models folder.
Select HybridIntelligence-0.5B-Q4_K_M.gguf from the model menu.
Use raw text completion or standard system/user templates.

sizzlebop/HybridIntelligence-0.5B-GGUF