I wanted a model to teach me Japanese with natural voice conversations. Through trial and error found out that you can only quantize layers that do not feed to talker, else voice will be garbled. Enclosing inference script + yaml, quantization script, modified llmcompressor and a proof of concept multiturn conversation script in extras.