Unfortunately, the lack of an input cache discount makes it prohibitively expensive for most use cases that aren't one-shot prompts.

well same applies to GPT OSS 120. Qwen is just the much smarter model of the 2 public options on Cerebras.

The point is the speed.

If the model provides me with bad results because it's dumb, I don't care how quickly it does it.

But there are lots of use cases where a relatively "dumb" model is good enough.

Is it possible to construct a control system where bad, fast and cheap can become good, fast, and cheap through repeated sampling and a strong spec/eval harness?

I am trying to keep an open mind with AI, but I also have little understanding of control theory, trying to learn.

You can, but you need to break the problem into much smaller tasks, then check those answers, and finally have a harness that handles all the context, task breakup, task definitions, and validations each round.

Smaller models seems to get stuck in "loops" when you try to "handle" them this way.

fast results that you need to verify are better than slow (allegedly better) results that you still need to verify. REPL vs batch.

> It is, but isn’t it all warranty in your scenario?

There are numerous recent stories of companies refusing to make good on their warranties and instead offering customers a refund for the original price paid. One example: https://www.tomshardware.com/pc-components/hdds/toshiba-refu...

Because the end goal is to ban non-US AI companies from being able to do business in the US.

...because everyone saw how well that worked for the Jones act, what with all the naval yards the US has lost over time, and how nearly no US-built ships operate where not legally mandated /s

Just because it's a bad idea, doesn't mean they won't do it.

some US companies benefit from that act for sure.

For caching, only if you don't specify your preferred providers and let OpenRouter route each request itself. I have stuff like this in my OpenCode config for each model I use and I regularly get ~90-95% cache hit rates.

    "order": ["relace", "coreweave", "novita", "baseten", "together"],
    "allow_fallbacks": false

It still won't be quite as high as you'd get by just using DeepSeek because occasionally a request will fail and you'll get routed to a backup provider with nothing cached, but it's close enough not to matter in most instances.

But I can't argue with the lower off-peak pricing when using DeepSeek directly. The downside is they train their models on your input, which might be a deal-breaker for many users (as it is for me).

Honestly, I know several devs who do chores around the house or even play video games while AI does the bulk of the heavy lifting. If nobody cares or even realizes, does it matter? (To be clear, they all work remotely.)

Which doesn't matter if they're still not profitable.

I don't see any kind of input cache discount listed on your pricing page. Do you offer that, or is all input priced the same?

The issue is that all input (including context) counts towards that limit. So 10 requests with 50k of context will blow through the limit, even if little to no output was generated, which is incredibly easy to do with agentic workloads.

This is pretty terrible advice when there are dozens of AI inference providers out there serving great models with significantly more cost effectiveness than you'd get from buying your own hardware.