wifuGPT-1.7B GGUF

GGUF quantizations of n0ctyx/wifuGPT-1.7B. For use with Ollama, llama.cpp, LM Studio, KoboldCpp, etc.

| File | Quant | Size |

|------|-------|------|

| wifuGPT-1.7B-Q4_K_M.gguf | Q4_K_M | 1.1 GB |

| wifuGPT-1.7B-BF16.gguf | BF16 | 3.3 GB |

Download

For the best experience, download using huggingface-cli and then create the Ollama model from the local file:

pip install -U huggingface_hub
hf download n0ctyx/wifuGPT-1.7B-GGUF wifuGPT-1.7B-Q4_K_M.gguf --local-dir .

Ollama

Create a Modelfile:

  FROM ./wifuGPT-1.7B-Q4_K_M.gguf
  PARAMETER temperature 0.7                                                                                                                                             
  PARAMETER top_p 0.8
  TEMPLATE """{{- range .Messages }}                                                                                                                                    
  <|im_start|>{{ .Role }}                                   
  {{ .Content }}<|im_end|>                                                                                                                                              
  {{ end }}                                                 
  <|im_start|>assistant                                                                                                                                                 
                                                     
                                                                                                                                                                        
  
  """

  ollama create wifugpt -f Modelfile                                                                                                                                    
  ollama run

llama.cpp

./llama-cli -m wifuGPT-1.7B-Q4_K_M.gguf -p "Hey!" -n 512

| Variant | Repo | Size |

|---------|------|------|

| bf16 safetensors | n0ctyx/wifuGPT-1.7B | 3.4 GB |

| 4-bit safetensors | n0ctyx/wifuGPT-1.7B-4bit | 1.35 GB |

| GGUF (this) | n0ctyx/wifuGPT-1.7B-GGUF | 1.1 / 3.3 GB |

Disclaimer

Contains explicit adult content. Use responsibly.

pssst~ wanna help out? 💕

hii so um... making these models takes like SO many GPU hours and my wallet is literally crying rn 😭 between abliterating, generating datasets, fine-tuning, and quantizing -- it all adds up super fast (,,>_<,,)

if this model made you smile (or blush~) maybe buy me a coffee?? it keeps the GPUs warm and the models coming ✨

![Buy Me A Coffee](https://buymeacoffee.com/n0ctyx)