Model Card for Self-Healing-LLM (Project RetroEdit)

Self-Healing-LLM is an experimental adaptation of Qwen/Qwen2.5-0.5B designed to overcome the strict forward-only constraint of standard autoregressive generation ($P(x_t \mid x_{<t})$).

Rather than treating output as an immutable sequence of tokens, this system introduces Editor Control Tokens (ECTs) paired with an inference runtime supporting Dynamic KV-Cache Splicing. This allows the model to backspace, rewrite, and repair prior lines in-place upon execution or syntax verification failures—avoiding costly full-context re-generation.

1. Token-Level Backtracking: Branching Probability

When backtracking is introduced, the model evaluates multiple potential candidates ($k$) at time step $t$. The probability of selecting a specific token $x_{t}^{(k)}$ depends not just on the history, but on a lookahead or evaluation function $V$ (often a value network or verifier):

$$P(x_{t}^{(k)}\mid x_{<t})\propto \exp \left(\text{logits}(x_{t}^{(k)})+\gamma \cdot V(x_{<t},x_{t}^{(k)})\right)$$

$V(x_{<t}, x_t^{(k)})$: An evaluation metric (like Monte Carlo Tree Search or a critique prompt) that scores how promising this path looks.
If the score drops below a certain threshold during later steps ($t+1, t+2$), the model backtracks to $x_{<t}$ and samples a different token with the next highest probability.

2. Sequence-Level Search: Tree/Graph Search

Instead of a single linear probability, a backtracking LLM optimizes the probability over an entire search tree (such as Tree of Thoughts (ToT) or Reasoning Graphs).

The goal is to maximize the success probability of the final sequence $X$ by searching through valid states:

$$P(\text{Success}\mid x_{<t})=\sum {x{t}}P(x_{t}\mid x_{<t})\cdot P(\text{Success}\mid x_{<t},x_{t})$$

If $P(\text{Success} \mid x_{<t}, x_t) \approx 0$ (the current path hits a dead end or logical error), the backtracking mechanism prunes this branch and returns to the parent state $x_{<t}$ to evaluate $P(x_t' \mid x_{<t})$.

🔄 Comparison: Standard vs. Backtracking LLM

Feature	Standard LLM ($P(x_t \mid x_{<t})$)	Backtracking LLM
Path	Linear. Greedy or nucleus sampling moves strictly forward.	Tree/Graph. Explores multiple paths, rewrites, and corrects course.
Mistake Handling	Hallucinates or compounds error. Cannot undo a generated token.	Self-corrects. Erases low-scoring tokens and restarts from a stable state.
Inference Strategy	Chain of Thought (CoT).	Tree of Thoughts (ToT) / MCTS (like OpenAI's o1/o3 architectures).

Model Details

Model Description

Developed by: Ashish Kamble (kambleaa007)
Model type: Causal Decoder-Only Transformer (with non-monotonic decoding logic)
Language(s) (NLP): English, Python (AST-targeted code)
License: Apache 2.0
Finetuned from model: Qwen/Qwen2.5-0.5B

Model Sources

Repository: https://huggingface.co/kambleaa007/Self-Healing-LLM

Uses

Direct Use

In-Place Self-Healing Code Synthesis: Generating scripts and functions where intermediate line syntax is continuously verified via an AST or linter, rolling back and repairing invalid lines directly.
Dynamic KV-Cache Research: Exploring random-access KV-cache slicing, token eviction, and non-monotonic generation workflows.

Downstream Use

Integration into IDE extensions, Language Server Protocol (LSP) engines, and agentic coding sandboxes requiring targeted error recovery without context-window bloat.

Out-of-Scope Use

Production deployments requiring strict deterministic outputs without execution sandboxing.
Direct standard pipeline usage (pipeline('text-generation')) without the specialized non-monotonic decoding loop, as standard pipelines do not handle mid-stream cache truncation.

Bias, Risks, and Limitations

Execution Latency: Verification checks (e.g., AST parsing) add compute overhead during token generation.
Syntactic Scope: Built-in verification is tuned for Python parsing rules; other programming languages require tree-sitter or grammar-specific validators.
Context Recalibration: Splicing within arbitrary sequence ranges can cause positional embedding drift unless paired with relative positional encodings or explicit RoPE adjustments.

How to Get Started with the Model

Standard Hugging Face generate() runs monotonically. Use the custom inference loop below to execute in-place self-healing with dynamic cache truncation:

import ast
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers.cache_utils import DynamicCache

MODEL_ID = "kambleaa007/Self-Healing-LLM"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto" if torch.cuda.is_available() else "cpu"
)

def trim_kv_cache(past_key_values: DynamicCache, target_len: int) -> None:
    """Crops the DynamicCache safely across all layers."""
    if hasattr(past_key_values, "crop"):
        past_key_values.crop(target_len)
    else:
        for layer_idx in range(len(past_key_values.key_cache)):
            past_key_values.key_cache[layer_idx] = past_key_values.key_cache[layer_idx][:, :, :target_len, :]
            past_key_values.value_cache[layer_idx] = past_key_values.value_cache[layer_idx][:, :, :target_len, :]

def is_valid_or_partial_code(code_str: str) -> bool:
    """Checks whether the code is valid or a valid partial prefix."""
    try:
        ast.parse(code_str)
        return True
    except SyntaxError as e:
        if "unexpected EOF while parsing" in str(e) or "expected an indented block" in str(e):
            return True
        return False

def generate_self_healing(prompt: str, max_tokens: int = 80, max_retries: int = 3):
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    token_stream = list(inputs.input_ids[0].cpu().numpy())
    prompt_len = len(token_stream)

    past_key_values = DynamicCache()

    with torch.no_grad():
        outputs = model(input_ids=inputs.input_ids, past_key_values=past_key_values, use_cache=True)
        past_key_values = outputs.past_key_values
        current_token = torch.argmax(outputs.logits[:, -1, :], dim=-1).unsqueeze(-1)

    line_checkpoints = [len(token_stream)]
    retry_count = 0

    for _ in range(max_tokens):
        tid = current_token.item()
        token_text = tokenizer.decode([tid])

        if "\n" in token_text:
            line_checkpoints.append(len(token_stream))

        token_stream.append(tid)

        if "\n" in token_text and len(line_checkpoints) > 1:
            current_code = tokenizer.decode(token_stream, skip_special_tokens=True)
            if not is_valid_or_partial_code(current_code) and (retry_count < max_retries):
                retry_count += 1
                last_valid_len = max(prompt_len, line_checkpoints[-2])
                
                # Truncate KV Cache and rewind token stream
                trim_kv_cache(past_key_values, last_valid_len)
                token_stream = token_stream[:last_valid_len]
                line_checkpoints.pop()

                prev_token = torch.tensor([[token_stream[-1]]], device=model.device)
                with torch.no_grad():
                    redo_outputs = model(input_ids=prev_token, past_key_values=past_key_values, use_cache=True)
                    past_key_values = redo_outputs.past_key_values
                    # Sample probabilistically to break deterministic error loops
                    scaled_logits = redo_outputs.logits[:, -1, :] / 0.8
                    current_token = torch.multinomial(torch.softmax(scaled_logits, dim=-1), num_samples=1)
                continue
            else:
                retry_count = 0

        with torch.no_grad():
            outputs = model(input_ids=current_token, past_key_values=past_key_values, use_cache=True)
            past_key_values = outputs.past_key_values
            current_token = torch.argmax(outputs.logits[:, -1, :], dim=-1).unsqueeze(-1)

        if tid == tokenizer.eos_token_id:
            break

    return tokenizer.decode(token_stream, skip_special_tokens=True)

# Example Execution
code_prompt = "def calculate_discount(price, rate):\n    # Return discounted price\n"
print(generate_self_healing(code_prompt))

## Training Details

### Training Procedure

* Extended base tokenizer vocabulary with custom control tokens (`<UNDO>`, `<SEEK_LINE>`, `<EXEC_CHECKPOINT>`).
* SFT fine-tuning on synthetic mutation-repair and diff-stream trajectories.
* Evaluated on syntax-preserving line rollbacks using Hugging Face `transformers.Trainer` and `DataCollatorForLanguageModeling`.

#### Training Hyperparameters

* **Training regime:** fp16 mixed precision
* **Learning rate:** 2e-4
* **Optimizer:** AdamW
* **Warmup steps:** 10

---

## Technical Specifications

### Model Architecture and Objective

* **Base Architecture:** Qwen2.5 (Decoder-only Transformer)
* **Parameters:** ~494 Million
* **Context Length:** 32,768 tokens
* **Positional Embeddings:** RoPE (Rotary Position Embeddings)
* **Attention Cache:** Paged dynamic KV-cache truncation compatible with `transformers.cache_utils.DynamicCache`

---

## Citation

```bibtex
@misc{kamble2026selfhealingllm,
  author = {Ashish Kamble},
  title = {Self-Healing-LLM: Non-Monotonic Decoding and In-Place KV-Cache Splicing for Code Synthesis},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{[https://huggingface.co/kambleaa007/Self-Healing-LLM](https://huggingface.co/kambleaa007/Self-Healing-LLM)}}
}