I had a lot of fun, the scan line effect looks really good and the obstacles are interesting and challenging. I'd have liked to use my game controller, there's a browser API for it. And to have changed my control settings. I thought both mobile and desktop controls were good though.
I've been working on a browser game too lately. It's a pretty great environment to work in. Three.js + WASM have cured my desire to learn Godot, at least for now.
If this hypothesis were true then a spy agency may want to rewrite responses. Every tool in an agent's harness becomes an remote procedure call you can make on that machine. Including a tool to execute a shell command, in many. A harness is completely isometric to a backdoor, it's the same code written with a different intention.
Ad-hoc hypothesizing ("escape hatches") are dangerous but not quite invalid. Eg, they failed to find gravitational waves until they did, and you could have viewed building yet another more sensitive detector as a similar exercise in refining a hypothesis that you keep receiving contrary evidence for. Sometimes you really did just underestimate how difficult your hypothesis was to demonstrate. Maybe Meta will be destroyed in 2027, or whatever.
The problem with conspiracy theories is more that they have a ratchet-like quality where counter evidence reaffirms the theory in your view and you can only ever get more confident. We should have been increasingly skeptical of gravitational waves to some degree as we failed to demonstrate them, even though we didn't abandon the hypothesis and it ultimately prevailed. But if you adopt a wrong idea, and people try to demonstrate that to you, and you take that effort they're putting forward as a sign that you are correct and they must be hiding something from you, it will be very difficult for you to realize your mistake.
So, as long as you are less certain than you were before, I don't think rolling your prediction over into the future is necessarily conspiratorial or a mistake.
I don't think you can extrapolate that measurement across multiple sequential draws like that. We presumably are comparing against a single trajectory rather than a tree of trajectories. So once we make the wrong choice and step off of the blessed path, we have no way to assign a ranking to the next token; it's error is undefined.
I've seen LLMs self correct in chains of thought ("because of foo and bar, I need to... Wait, bar is not true, so that won't work") so I have to imagine this is a massive overestimate, errors do not necessarily compound.
I would say they do compound until proven otherwise.
Having "Wait, bar is not true, so that won't work" is not necessarily a correction. In fact, the problem is: across a long text it is a correction of a single mistake, but we are talking about thousands here.
But yes, of course that was a rough estimate. But the problem is - we don't really know what we are measuring here. Maybe there's a 2,000,000x difference of intelligence between coding indexes 52 and 50. By some measure that just feels small because that's how we process it akin to audio db.
Regardless the point is KLD and whatever they came up with is not meaningful. And they did not publish comparisons on real benchmarks.
Just at a sniff test level, don't you think that if the quantization resulted in anything like 2M% error in a pretty typical context length, it would be plain as day? You'd do an A/B test and one of them would look like standard generated text and one of them would veer into incoherence? If not - what on would 2M% error even mean then?
> Regardless the point is KLD and whatever they came up with is not meaningful.
I'm not saying you're wrong, I'm just saying this isn't a meaningful metric either, mostly because it is using a different type of error (divergence along a trajectory) than what was actually measured (divergence at a fixed point) and so can't be used for this purpose. It could establish an upper bound but going by your work that upper bound is so high it may as well be infinite. That's somewhat concerning but doesn't necessarily suggest it performs badly in a typical case (which is how I'd interpret an expectation of 2M% error).
It doesn't really matter to this argument, if they are self correcting at all, then we can't assume all errors will permanently injure the trajectory. It's not like dead reckoning or a similar process where there is never an opportunity to reassess. It's more like a long division problem; it is possible to correct errors using in band information, without the external reference dead reckoning would require. (The incidence of false positive self correction does matter to the question of whether the model is actually of comparable quality after the quantization, of course.)
You can see bathtub rings on the underside as it's lifted up. The rings are caused by it's melting faster under the water than in the air, so there's a discontinuity at the waterline.
> If the agent itself concatenates a new URL together - with leaked data after a ? - you should block that from being fetched.
You're correct of course, I just want to note that the exfiltrated data could be in any part of the URL, so the absence of a query string doesn't indicate that no payload has been encoded into the URL. Arbitrary example, you can include credentials in a URL, so you could encode the exfiltrated data into a password.
If 1 in 10,000 people have a disease, then a "test" which always reports the patient doesn't have the disease will be correct 99.99% of the time. "99.99% accuracy" should be "looked at with skepticism" in that it doesn't tell you what you need to know to understand the quality of a a test for a rare disease (a classifier under conditions of severe class imbalance); at a minimum, you would want to understand it's false positive and false negative rate, not (just) it's overall error rate.
You appear not to have understood probability theory my friend. You will never get 100% in this universe for anything. What if "its a simulation" or "a dream" arguments ensures you never acheive 100%.
Bayes probability theory will be a good start for you.
Well, I do agree that all measurements contain error, but the point wasn't that the error rate would be greater than 0% but that a single headline summary of error can't always distinguish between good and bad tests.
Claude Code is quite buggy but doesn't generally crash, which is what you would expect if it shipped an immature backend for a month. Maybe the rewrite is full of bugs and they happened to result in the UI glitches or trashed settings files or whatever other application-level bugs that Claude Code has routinely rather than crashes but that would be pretty surprising.
I've been working on a browser game too lately. It's a pretty great environment to work in. Three.js + WASM have cured my desire to learn Godot, at least for now.
reply