A merged vLLM patch compresses Qwen3.6's 16-bit lm_head from 970 MB to 300 MB at load time, cutting median decode time by 2.82 ms and gaining 12% on Strix Halo, with the perplexity cost published.
Read it at RuntimeWire →
← Front page