Exl3 5.0bpw h6 quantization of Qwen/Qwen3-235B-A22B-Thinking-2507 using 69750c8 commit of Exllamav3
148.4gb, requires ~160gb with 64k unquantized context.
Exl3 5.0bpw h6 quantization of Qwen/Qwen3-235B-A22B-Thinking-2507 using 69750c8 commit of Exllamav3
148.4gb, requires ~160gb with 64k unquantized context.