rnj-1.5-instruct-GGUF
GGUF conversions of pszemraj/rnj-1.5-instruct, a sliding-window-patched repack of EssentialAI/rnj-1.5-instruct. See the upstream card for architecture, training, and benchmarks.
Files
| Quant | File |
|---|---|
| Q6_K | rnj-1.5-instruct-q6_k.gguf |
| Q4_K_M | rnj-1.5-instruct-q4_k_m.gguf |
[!IMPORTANT] The base repo substitutes sliding-window attention for the original block-local attention. Equivalent to upstream for prompts 8192 tokens; diverges beyond and YMMV past 8192
Usage
llama-cli -hf pszemraj/rnj-1.5-instruct-GGUF:Q6_K -p "how to cook minute rice in 58 seconds?"
llama-server -hf pszemraj/rnj-1.5-instruct-GGUF:Q6_K -c 8192
Swap :Q6_K for :Q4_K_M for the smaller quant. Proof it works: