If you’re on AMD, use llamacpp with the vulkan backend. For the 35B I get around... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

mhjkl 46 days ago | parent | context | favorite | on: Kimi-K3 on HuggingFace

If you’re on AMD, use llamacpp with the vulkan backend. For the 35B I get around 115 tps without mtp and 150-170tps with it. Also the pcie slot you use affects it

ThunderSizzle 46 days ago [–]

I have issues with the vulkan backend. It seems to cause over all GUI sluggishness, but maybe that was actually Firefox. I'll give it another go.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact