I quantized the base model using compressed tensorers + big calibration dataset and re-added MTP weights from the original Qwen model as they were missing in heretic model. Two token MTP prediction seems to work fine with decent acceptance rate. I have included compression script, vllm inference script and MTP copy script in experts. To get VLLM working I had to disable cuda graph compilation and install conch-triton-kernels. After this I get about 15tps peak generation speed on my NVIDIA Thor dev kit. If you get various exceptions, take a look at inference script and copy various tuning parameters.