4-bit Quantization of the Kimi Dev 72B model

Quantized using GPTQModel

quantiziation config:

quant_config = QuantizeConfig(
    bits=4,
    group_size=32,
    sym=True,
    desc_act=False,
    true_sequential=True,
    pack_dtype=torch.int32,
    damp_percent=0.1       
)