It's something that's been bugging me for a while: once we reach a "good enough" small model, and qwen 3.8 27b is already getting damn close to it, does it make sense to just bake weights and everything directly into an ASIC, and use that for highly optimized inference? AFAIK only groq and cerebras are moving in that direction, and only to be providers themselves... It would be a dream to buy one for <$1k.
This is exactly what Taalas has already done, and the reason they got quickly acquired by AMD. Their chip runs Llama 3.1 8B at 17k tok/s, and even if llama 3.1 is dated, I can think of many problems I could use it for, especially at those speeds. They're certain to be working on a newer set of weights by now.
