Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.


I've had it run to ~400k when debugging "obscure" (to it) sequences. Wouldn't recommend more.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: