Probably they bias the RNG for selecting the next token. This can be done practically in a lot of ways, including during training.
I suspect the signal will be significantly under the noise floor, so it's not detectable if you don't know exactly what to look for, but certainly you can submit more information then the textual contents.
But if the user's prompt is in the context, you don't know exactly what the RNG chooses between. I don't know what trick they use to get past that, but it seems impossible to get by it in the general case (i.e. if the prompt can be anything) and you'll probably quickly compromise quality if you try.
A reasonable guess about the algorithm is 'A Watermark for Large Language Models' (https://arxiv.org/abs/2301.10226). The idea is that each generated token (or bigram) seeds a strong PRNG that splits the vocabulary into a 'green' and 'red' set. The sampler then tries to select a 'green' next-token for generation.
After-the-fact checking only needs the vocabulary splitter, which is independent of the LLM. Over a sufficiently large text non-watermarked text would expect to use green and red tokens with the baseline probability, and that difference can easily become statistically significant over sufficiently long texts.
The basic algorithm has obvious knobs to tune, among them the initial ratio of red to green tokens and how hard the sampler tries to pick a green token. These would balance fidelity to the original distribution against watermark detectability (minimum required content length for statistical power).
Anthropic actually tells you the approach they use, and it's not that. From their Claude Text Watermark page[0]: "Claude’s text watermark is a version of the SynthID-Text approach published by Google DeepMind in a Nature paper in 2024."
The Nature paper is "Scalable watermarking for identifying large language model outputs"[1]. This method does not separate out tokens into separate classes, but merely uses a seed for the PRNG that selects which among the most likely tokens generated by the LLM will actually be output. This has the advantage that there's no green and red token sets, so no token is systematically favored or disfavored. If a particular token is overwhelmingly predicted to be the most likely candidate, it will almost certainly be selected, so the watermark doesn't affect that. Even if there are several choices of output token at a point that have similar probability of selection, the watermark doesn't systematically bias in favor of one token or the other.
This is actually a quite elegant method of watermarking that, contrary to people's fears, won't adversely affect the model output. The main concern I have with it is that it appears that you can't actually test the watermark locally, without uploading it to Anthropic. I'm not sure why that's the case, since there's no particular reason the watermarking key has to be private, except if you want to prevent others from generating text with their own LLMs that is watermarked to look like it's generated by Anthropic - but everybody wants their text to not have the watermark.
I just got Claude to download and install all the models and servers and agents and prepare all the launch scripts for me... no need to learn, just ask it to do it for you
Right, but I am a middle-aged bloke who is experiencing existential angst about whether I can carry on in this industry.
I have a pretty deep, maybe paranoid need to be confident I have an intrinsic understanding, and I have found in my life that lessons come to you when you make yourself open to learning.
So I need to build on top of what I know, taking as much of the hard way as I can bear to take at any one time — it has to be not quite difficult enough to put me off.
I can't really explain what I have learned this way that is different, but I feel it in a way that I wouldn't if I'd simply pushed a button.
For the same reason, I have a really basic 3D printer that I've set up myself, set up Klipper, configured how I want it, learned how to calibrate, all that. And now I can say that I feel I have an understanding of 3D printing. I could hold my head above water in a discussion with a real expert, maybe find work in an adjacent field where my insights would keep me grounded.
I can afford a really good printer that has all that set up, and more, has no problems. But I'd just be someone who has a 3D printer.
(Also who am I kidding about the existence of a printer with no problems)
This really resonates with me, and I'm only a decade and change into my career. I use claude a lot day to day. I try to use it sensibly, making me more productive and produce better work. I'm also trying not to lose understanding along the way. I want to be able to actually talk to the conclusions I'm reaching.
I have colleagues that seem perfectly content to delegate too much to the agents, and it saddens me. It feels like there will be swaths of engineers that didn't train some of the critical thinking skills that I take for granted.
I certainly see it in slack discourse around anything more complicated than a feature implementation. Maybe I'm just cynical. Time will tell, I suppose.
You will not live enough to learn everything. Eventually you have to say "I could figure [something] out but I won't take that time." Most things are that way - I probably could learn brain surgery (I used this example because it has a reputation of being a very difficult course of study). I would like to make a lathe from scratch - but I don't have easy access to enough iron ore to get started - even if I start from scrap metal, I probably wouldn't spend months making my own surface plate (...) and so I own a factory made lathe instead.
That is why I'm content to delegate to agents - I have more code/features I want to write than I have time to debug (writing is the easy part).
For me (about halfway between you and dofm in my career by your own statements in this thread), it's a dream at the moment. I can delegate all the tedious stuff that I've done "the hard way" a thousand times already and feel I have very little of value remaining to learn, so that I can spend more time on all the things that are actually new and thus much more interesting.
It's been a great multiplier for me in similar ways. The "dreamiest" thing has been that it has freed up time that I would normally have spent doing sprint work, to work on things that just don't make the cut until it's bad enough to deprioritize other work.
Over the last few months, I've been digging into performance problems with a high throughput service that my team owns. I started working on the problems in my own time, put out short and medium term improvements that legitimately avoided operational issues, and started developing an alternate architecture that should meaningfully address the problems for the long term.
I've learned new things and made improvements that probably wouldn't have ever gone in otherwise.
Yes exactly. There is a narrative that it's driving everything toward low quality slop, but in my own work it's exactly the opposite. We're doing work on quality and performance that we never would have gotten to in the past.
I've spent my whole career being frustrated by the pile of low severity bugs and performance issues that "I could fix that if I could only justify putting a couple hours into it!". And now I can just fix all those. Nobody is going to question my use of time to write prompts and do code reviews of those things, when I can to my "real" work simultaneously.
Yeah, this is just the engineer's mindset. It's not surprising that this is a popular view here, even if it is not (and does not need to be) the mainstream perspective.
This is a very fair question! When I wrote this comment, I was definitely thinking of the "real" mainstream, i.e. users of llm chat to generate text, not software engineers.
But I think there is (and has always been) also a distinction between the "mainstream" of software developers vs people who are working on new tools and capabilities to be used by that "mainstream".
IMO it is certainly true that the most efficient and cost effective was to do "mainstream" software delivery at the moment is hosted frontier models. But for people thinking about "what's next?", it makes a ton of sense to be exploring different models in anticipation of a possible (but certainly not inevitable) sea change.
I don't necessarily think your answer is wrong for all people, but if you work in software... how do you plan to differentiate yourself from everyone else out there, if the depth of your understanding is "Claude can do it for me"?
I mean one of the things I use a local LLM for, because I can, is to generate starter documentation. But I ask it to — I want it to give me overviews, plans, all that. It can make something bespoke for me.
I guess I could also ask it to do the work. But where do you draw the line?
The universal labour-saving device is the great provocation of the next 100 years I think, and both Star Trek and Wall-E have grappled with it.
The skill isn't the plowing. The skill is thinking and learning and the ability that atrophies is that of mental effort (which is what drives thinking and learning). Losing those will affect people's lives and potentially even their humanity.
The reason I delegate so much of local LLM installation and administration to Claude Code is simply because there's no point learning practical things that will work completely differently in a couple of years, or in memorizing procedures that I'll forget long before I need to perform them again.
No longer having to sweat all the details is a Good Thing, not a Bad Thing.
I am not sure I disagree, and I certainly don't mean to disagree very fervently.
But I think if you want to really learn to ride well, understand horses well, there might be some benefit in learning how to shoe a horse. At some level it should never only be someone else's job.
You actually do need some understanding of how a car works, no?
For example, you need to know it uses gasoline (or diesel), it requires oil changes every certain amount of time, break pad replacement, etc.
You also probably need to know that you can't operate cars over a certain amount of water, that you need a driver's license, stopping at red lights, etc.
Sure, you might not need to be a mechanic, but that's far from not understanding how a car works, which to me sounds similar to knowing how to shoe a horse, which is different than being a horse vet.
Yes, LLM are thrown through pretty much everyone digital life whether they like it or not, it's not just devs. It might even unlock exploring things that need code that average user wouldn't have dared to do before.
It did not. There are professional horse shoers like there always were. Not in huge numbers, but there are. Domesticated horses didn't just disappear from the world.
I think if you really don't feel the need to know the "why" of everything, sometimes this might be the right approach. It is quick, pragmatic, gets you started.
Maybe my biggest problem with the world of agentic AI, and the reason I am putting myself through learning it the way I am, is that the need to know the "why" of everything is so fundamental to me, that I don't know if there is any point to me without it.
So this is really the only way I know how to proceed.
To me, this is just a question of specialization. Not everyone needs to be a "I understand how the system actually works" person. In fact, not many people need to be that person. But every system does need some of that person to exist!
And we happen to be discussing this on a forum where the type of people who will be the specialists for the kinda of systems we're discussing are likely to gather.
I'd be surprised if in my casual discussions out in the real world, I were to run into a lot of people who care exactly how all this works, to the extent that they want to invest significant money into hardware that allows them to run things themselves and dig into what's actually going on. But I'm not at all surprised to come across such people here! (Indeed, it would be very disappointed if I didn't!)
I think the more you know of how (many) things work, the slightly better you'll be at using them. From dishwashers to CPUs, from car engines to watercolours, from guitars to kitchen knives... You get the gist. Once you internalize a model of the thing, it becomes closer to an extension of you than a tool. You drive it better and with less friction.
Yes agreed, but there is limited time in a life, so there is a fairly high opportunity cost to internalizing a model of many things, which scales quickly with the complexity of those things, so people rationally limit the number of things they invest their time in. For the vast majority of people, I think it makes a lot of sense for AI systems to fail to make this cut. But for most of us here, on a site for computer technologists, it almost certainly makes sense for us to learn as many of the details as we can manage.
Why should they ?! Do you also want to force them to design their cars so the engine is easily replaceable by a Custom Engine OS so that the community can build their own engines ?!?
Because laws are (mostly) a reflection of what society wants.
People are growingly concerned with both the car manu and Apple/Google control over their car and related extra software goodies.
Laws are really needed when businesses don’t play nicely. I don’t know the legal specifics, but I’m sure glad I don’t need to buy $1000’s of specialty tools to maintain my vehicle, and sure glad that replacement parts are readily available (and will be for decades).
Just image how much worse society would be if car manus did the same thing as Apple and had ID-paired parts. Sorry! Your AC doesn’t work anymore, please install a genuine Honda oil filter at your nearest Authorized Honda Shop, available for a minimum of $500.
> People are growingly concerned with both the car manu and Apple/Google control over their car and related extra software goodies.
10 out of random 10 drivers out there don't care about the software running in the car.
> Laws are really needed when businesses don’t play nicely. I don’t know the legal specifics, but I’m sure glad I don’t need to buy $1000’s of specialty tools to maintain my vehicle, and sure glad that replacement parts are readily available (and will be for decades).
You drive a self-maintained car. Nothing wrong with that, but I would guess 95 out of 100 drivers on the road don't care about the car at all - they just want reliable transportation from A to B and perhaps some confort.
> Just image how much worse society would be if car manus did the same thing as Apple and had ID-paired parts. Sorry! Your AC doesn’t work anymore, please install a genuine Honda oil filter at your nearest Authorized Honda Shop, available for a minimum of $500.
I don't have to imagine that al all, all premium car manufactures digitally id their components and will not accept 3rd party replacements.
Honda isn’t a premium car manufacturer, but also can you please show a source for “all premium car manufactures digitally id their components and will not accept 3rd party replacements.” I know plenty of mechanics and while OEM is usually the way to go there are many aftermarket parts that get installed regularly.
I would guess there are a couple of orders of magnitude difference between the complexity of interfaces comparing the head unit with wheels and tires.
Like, the head unit is in control of all that happens on the slow bus of the car, and needs to pass independent safety certifications for a complex system.
Fuck that, we're on this boat together, and we'll all go to the bottom together.