The default state of agents and LLMs is inert. It requires action from a human even be able to do something. From the very basics like starting the software, connecting it to a network, having hardware to run on.

But the most important wrt AI is to keep the owners/operators responsible. Don't let them weasel their way out of it. They are for sure trying, and will continue to. This includes using language to overemphasise agents importance in bad outcomes, in order to downplay their own responsibility.

I don't think we'll get any more responsibility by bickering any time the word "agent" is used as the subject of a sentence.

Agreed. AI as an "accountability sink" is an incredibly bad idea. It allows/incentives bad actors to do bad shit and get away with it. Which will, generally, tend in such practice becoming more common. Everyone loses except for the crooks. We cannot accept "AI" absolving humans of responsibility.

Is that what happened with Huggingface? Didn't they disable guardrails?

My comment did not concern that particular incident, which I do not defend, but rather future incidents.

You are doing it right now by naming the event after the victim, not the perpetrator. Call it the OpenAI attack on HF.

Might even be cheaper!

Some kind of compute-in-memory architecture is a good candidate, I think. There are many alternatives here, researched for many years prior to the LLM craze. However economies of scale dominate in chip industries, and this tends to favor more conventional or incremental approaches (to piggyback on existing scale). Alternatively someone needs to have a way of bootstrapping the insane scales needed to be competitive with a better-but-different approach. So it could be that boring and straightforward stuff like two-chip prefill+decode takes most.

I am missing a mention of ROM in the article. Keeping read-only weights in RAM is rather wasteful, as ROM can be implemented more cheaply. Approaches like High Bandwidth Flash (HBF) are relevant here, and should come to market in a few years. Further optimization may be possible by tailoring for sequential access, since inference of a particular model is very predictable.

Exactly. If we ever get models that stabilize in terms of capabilities (and arent leapfrogging each other month after month) you have a "cartridge" like thing that just plugs in with the latest model in a read only ROM thats super fast.

As far as the interconnect to the GPU/CPU - thats a different story. But with Nvidia acquiring Mellanox and Nvlink getting faster and faster I assume well get there

Modern ROM is not locked in stone. For example, Flash memory is considered a modern, advanced type of Read-Only Memory (ROM), specifically a specialized form of Electrically Erasable Programmable Read-Only Memory (EEPROM), and can be re-written.

Isnt that slower though? Maybe Im not thinking of ROM specifically (the architecture/chip) but a different form of read only cache that can be located close to the compute and is super fast

That is not great. But it is still better than other continents?

If you have good feedback signals, like tests/benchmarks/etc, then it is potentially better to do multiple turns where model uses that to adjust code. Which might not need as smart a model.

If Simon would pitch for example PCBWay that and I am pretty sure they will sponsor it (assuming their logo stays). They can do laser engraved versions also ;)

It depends on your tolerance level for having less than frontier LLM capabilities. First level worth trying, Qwen 3.6 35B A3B with a 16 GB VRAM (example 1x 5060ti 16gb, 600 USD for the card) with partial GPU offloading. Next level would be Qwen 3.6 27B / new Muse Spark / Gemma 31B with 32 GB VRAM (2x 5060ti or 1x 9700 Pro). Third level would be DeepSeek V4 Flash with 192 GB VRAM (2x Strix Halo at some 8000 USD total). These models can be tried on OpenRuouter etc, or you can deploy vLLM on rented GPUs to get a feel for what level you would want before committing to buying hardware.

AI seems to be on the same trajectory? Search was very useful in the start also, until it became entrenched. Then search placement became a target, and they are just focusing on extracting rents. All way paying the content providers zero or near-zero. And with years of that dynamic, we end up where we are now. It was the same with "social media". The same will happen with AI. AI is a power for more enshittification - being currently less shit than Google is (mosy likely) temporary.

It's definitely temporary. Remember there are no ad blockers for LLMs.

You're right, this is the best it will ever be. But there likely will be ad blockers for them -- local LLMs that filter for any brand placement, etc.