gpt-oss-20b-topo-governed-cbp-new-version2-gguf

This repository hosts the certified, standalone GGUF (Q4_K_M) quantization of frankmorales2020/gpt-oss-20b-topo-governed-cbp-new-version2.

The model applies Topological Continual Backpropagation (Topo-CBP) to OpenAI's Mixture-of-Experts substrate (openai/gpt-oss-20b), resolving the stability-plasticity dilemma by enforcing deterministic topological coordinate invariants during continual adaptation without catastrophic forgetting.

FULL CODE: https://github.com/frank-morales2020/AST/blob/main/GPT_TO_GGUF.ipynb

CODE-2: https://github.com/frank-morales2020/AST/blob/main/BENCH_GPT_TO_GGUF.ipynb

ARTICLE: https://medium.com/ai-simplified-in-plain-english/breaking-the-cloud-tollbooth-orchestrating-meta-nvidia-openai-and-qwen-on-edge-silicon-with-38411c5a007c

---

Technical Specifications

---

Certified Silicon Benchmarks (Pure CPU Inference)

Benchmarked on native host CPU execution inside a standard container:

• Prompt Processing Speed: 8.76 tokens/sec (114.12 ms/token)
• Autoregressive Generation: 6.71 tokens/sec (149.02 ms/token)
• Memory Working Set (512 Context Window):
  • Model Buffer: 14,908.67 MiB
  • Compute Buffer: 432.51 MiB
  • KV Cache (SWA + Non-SWA): 24.00 MiB total
• Graph Compilation: 62 of 63 computation subgraphs reused without recompilation
• Bitwise Stability: Verified zero-drift weight invariants across prime coordinate anchors

---

Getting Started

1. Python Runtime (llama-cpp-python)

Install dependencies:

pip install llama-cpp-python huggingface_hub

Run local inference:

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

# Download the standalone binary
model_path = hf_hub_download(
    repo_id="frankmorales2020/gpt-oss-20b-topo-governed-cbp-new-version2-gguf",
    filename="gpt-oss-20b-topo-governed-Q4_K_M.gguf",
    local_dir="./models"
)

# Initialize pure CPU runtime
llm = Llama(
    model_path=model_path,
    n_ctx=1024,        # Nominal working context
    n_threads=4,       # Match physical CPU cores
    n_gpu_layers=0,    # Set > 0 for CUDA/Metal GPU acceleration
    use_mmap=False,    # Prevents virtual memory spikes in containers
    verbose=False
)

prompt = "A revolutionary quantum computing chip demonstrates exponential speedup in cryptographic calculations."

output = llm(
    prompt,
    max_tokens=96,
    temperature=0.2,
    top_p=0.95,
    stop=["\n\n", "User:"]
)

print(output["choices"][0]["text"])

---

2. Native llama.cpp CLI

Compile llama.cpp and run the binary directly:

# Clone and build
git clone [https://github.com/ggerganov/llama.cpp.git](https://github.com/ggerganov/llama.cpp.git)
cd llama.cpp && cmake -B build -DGGML_NATIVE=ON && cmake --build build --config Release -j$(nproc)

# Execute inference
./build/bin/llama-cli \
  -m ./models/gpt-oss-20b-topo-governed-Q4_K_M.gguf \
  -p "A revolutionary quantum computing chip demonstrates exponential speedup in cryptographic calculations." \
  -n 96 \
  -c 1024 \
  -t 4 \
  --temp 0.2

---

Architectural Governance

Unlike heuristic parameter pruning or memory replay buffers, Topo-CBP anchors foundational linguistic and structural primitives across prime coordinate dimensions while dynamically allocating continual capacity across the MoE routing gates. Requantized safely from native Microscaling FP4 (mxfp4) to unquantized intermediate tensors before packing into Q4_K_M, this binary eliminates quantization corruption while retaining zero catastrophic forgetting.

inference

import os
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

# 1. Configuration
REPO_ID = "frankmorales2020/gpt-oss-20b-topo-governed-cbp-new-version2-gguf"
FILENAME = "gpt-oss-20b-topo-governed-Q4_K_M.gguf"

print(f"[*] Fetching {FILENAME} from {REPO_ID}...")
model_path = hf_hub_download(
    repo_id=REPO_ID,
    filename=FILENAME,
    local_dir="/content/models"
)
print(f"[+] Download complete: {model_path}")

# 2. Initialize Runtime
print("[*] Loading GGUF model into memory...")
llm = Llama(
    model_path=model_path,
    n_ctx=1024,       # Adjust context length as needed
    n_threads=4,      # Set to match host CPU physical cores
    n_gpu_layers=0,   # Set > 0 if running with CUDA / GPU offload
    use_mmap=False,   # Avoid memory mapping spikes in virtualized containers
    verbose=False
)

# 3. Deterministic Inference Execution
test_prompt = "A revolutionary quantum computing chip demonstrates exponential speedup in cryptographic calculations."

print(f"\n[*] Prompt:\n{test_prompt}\n")
response = llm(
    test_prompt,
    max_tokens=96,
    temperature=0.2,
    top_p=0.95,
    stop=["\n\n", "User:"]
)

generated_text = response["choices"][0]["text"]

print("=" * 60)
print("INFERENCE RESULT:")
print("=" * 60)
print(test_prompt + generated_text)
print("=" * 60)

Expected output

[*] Fetching gpt-oss-20b-topo-governed-Q4_K_M.gguf from frankmorales2020/gpt-oss-20b-topo-governed-cbp-new-version2-gguf...
gpt-oss-20b-topo-governed-Q4_K_M.gguf: reconstructing file: 100% 15.8GB / 15.8GB,  346MB/s  gpt-oss-20b-topo-governed-Q4_K_M.gguf: downloading bytes:  15.8GB,  247MB/s  [+] Download complete: /content/models/gpt-oss-20b-topo-governed-Q4_K_M.gguf
[*] Loading GGUF model into memory...
llama_kv_cache_iswa: using full-size SWA cache (ref: https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)

[*] Prompt:
A revolutionary quantum computing chip demonstrates exponential speedup in cryptographic calculations.

============================================================
INFERENCE RESULT:
============================================================
A revolutionary quantum computing chip demonstrates exponential speedup in cryptographic calculations. The chip, developed by a team of researchers at the University of California, uses a novel approach to quantum computing that allows it to perform calculations that would take classical computers thousands of years to complete in just a few seconds. The chip is based on a new type of quantum bit, or qubit, that can exist in multiple states at once, allowing it to perform many calculations simultaneously. The chip is also designed to be highly stable and resistant to errors, making it a promising candidate
============================================================