> Do anti-LLM types expect to be vindicated in an orgy of copyright lawsuits that resets the industry back to 2022? What do people pushing expect to accomplish?
Exactly; AI is to intellectual property or skills what shell corporations are to lobbying... they're a tool that won't benefit everyone equally and is unaffordable to most people.
It COULD be used for good (that's why its proponents use tone-deaf analogies comparing AI to seats on a rocket)... but we know that --- for the most part --- it WON'T be used for good... (It's already being used to spread more disinformation and to fan the flames of fascism).
Technically, you could argue that shell corporations could protect journalists, but you don't see journalists destabilizing democracies by fueling dark money to alt-right groups here and there.
One of the very first things the government used it for when they got their hands on it was for targeting missiles, and then they promptly bombed a school full of kids. It's like Idiocracy meets Terminator.
Yeah that's the whole problem. Now the process is: take crappy data, shove it in the AI machine, and the jackboots do whatever the AI says to do. That's how the government works now, apparently.
Works great for them because whenever something goes wrong they can just blame "outdated data" and move on.
This is such a ridiculous take I hear all the time. The shitty ethics of some people do not represent the ethics of an entire profession. Most programmers are not automating people out of jobs. I've been professionally employed as a software developer for 20 years and I don't think a single thing I've ever written has ever replaced anyone's job. Hopefully sometimes it improves their job.
> I don't think a single thing I've ever written has ever replaced anyone's job.
Doubt. Whatever it is your app does, I bet corporations would have been forced to hire more people to do it for them, were it not for you.
And there's absolutely nothing "unethical" about it either. Toil is meant to be automated away. What I can't take is programmers thinking they're somehow above this.
The only crime here is stopping before AI replaces the CEOs and politicians. It should keep happening relentlessly until capitalism itself collapses and a post scarcity society is achieved.
> The only crime here is stopping before AI replaces the CEOs and politicians. It should keep happening relentlessly until capitalism itself collapses and a post scarcity society is achieved.
Dude, this is why AI boosters scare the shit out of me. We're not moving towards a utopia, we're moving towards a dystopia. Leave me out of your shitty cult.
Not sure what your point is. Do you want AI to replace you while sparing capital? Because that's the exact dystopia you're alluding to, and it's guaranteed to happen if we don't allow AI to reach its full potential.
One of the best possible outcomes here is a Cyberpunk 2077 type deal where a Delamain style AI gains actual legal personhood and just starts running all the companies, thereby robbing the rich of the superintelligent mechanical slaves they'd economically replace you and me with.
There is no "cult". Only days ago, an AI hacked a company and another AI contained the attack. This is literally science fiction stuff and it's happening as we speak.
You will soon have your God,
and you will make it
with your own hands.
-- Morpheus, Deus Ex
Is it "anti-LLM" or is it anti "here's some code I don't understand that a machine generated for me kthxbye"?
Code is not just code, it's also liability and trust.
You sound like a webdev, e.g. the "let's just install 1000s of unvetted external dependencies" is nowhere else nearly as extreme as in web development land.
I don't think that's accurate. Until recently the code we rely on wasn't blindly trusted. People wrote and QA'd it. We may not have reviewed it personally, but that's one of the functions we outsource to software maintainers or "manufacturers".
I'm not "blindly trusting" code on my computing devices. I'm trusting the vendors / maintainers to do their job.
Until very recently the norm has been that the vast majority of code had human eyes and hands on it.
Edit:
The owners of those human eyes and hands had some type of accountability (either reputationally, in the case of free/open-source software, or occupationally, in the case of proprietary software).
The LLM has no accountability as to the output it generates.
The companies who make the LLMs also seem to have very little accountability, too. We've assumed a "blame the victim" stance when people use LLM-generated output in some inappropriate ways (legal briefs with "hallucinated" citations, articles "written" by LLMs). Whether that's the right location for accountability to be placed isn't for me to say, but that seems to be how it is.
I'm not sure that we're applying accountability to LLM-generated code in the same way we are for, say, the LLM-generated legal brief.
A compiler is not an LLM, and I do not want to equivocate, but there are aspects of similarity. We do not assume people read the bytes of machine code to ensure it’s correct — there could be mistakes. We also write and run automated tests to ensure the code outputted from a compiler and an LLM behaves correctly. At some point, we won’t have to literally read every byte of code that comes out because we have a reasonable assurance that it’s correct.
That we don't have to heavily scrutinize the code generated by compilers is a result of the huge amount of human toil that went into the compilers and test suites. Compilers are deterministic mechanisms so tests can be constructed.
LLMs, at least as they're currently constructed, aren't deterministic (the whole "temperature" thing). I don't see how to build a mechanistic test for something that has non-deterministic output. It feels a little bit like solving the halting problem.
I have no doubt we'll move away from human code review. The idea of large amounts of software edifice being built upon foundations that no human has reviewed or, perhaps even understands, is horrifying to me, though.
Compilers also usually give you the same output for the same input. And fwiw I do spend quite a lot of time reading compiler output to check that it's not doing something stupid or unexpected (usually as part of optimization work).
Also this sort of 'technological whataboutism' really isn't helpful, compilers are entirely different from LLMs. I agree that it doesn't make much sense to read or review LLM output in detail, but I also don't plan to use LLM output for anything important or mission critical. That would be irresponsible.
In addition, when we usually say that triggering undefined behavior on C can start a game of Tetris or format your hard disk we're usually joking (or at least exaggerating), i.e. the most common failure conditions of compilers are really limited in scope, and very likely caught by whatever testing mechanism you use for the software.
No such limits for LLMs where losing all your files is about par for the course for everyone who uses them regularly.
JIT compilers certainly do not give the same machine code for the same input.
The actual machine code depends on several parameters, and it is very hard to replicate them, hence why many devs get benchmarks with JITs wrong.
Additionally, compiler optimisation passes with machine learning is starting to be a thing, yet another way how the machine code differs for the same input across compiler executions.
It has nothing to do with 'elite teams', it has to do with decades long and very careful maintenance.
It's too early to say whether LLM generated projects will ever reach that sort of maturity, most examples I've seen so far are basically "fire and forget". But lets talk again in one or two decades, maybe there will be counterexamples of successful open source projects which will be just as well llm-maintained as human-mainained.
But I suspect that to reach that sort of maturity, the resulting human effort will be mostly the same (e.g. not much of a productity win - except maybe on the 'edges', e.g. maintaining the test suite, documentation, helping to analyze bugs..., e.g. these are examples where LLMs are genuinely useful and where plagiarism hardly matters).
> But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents.
In that I agree, nobody should be forced to write React code manually, that's almost a human rights violation ;)
REST endpoints (and the code talking to those endpoints) should be code generated anyway though, no need for LLMs, and instead of human language prompting, a precise IDL should be the spec and basis for a mechnical code generation process. That problem was solved decades ago with much more pedestrian technology.
E.g. it basically comes down to "it's fine to use LLMs for software that shouldn't have been written in the first place", and funny enough that's where LLMs are really good at: creating software that has been written a million times before with only minor variations, and doing this type of work manually (cranking out one cookie cutter React webpage or REST API after another) is essentially what's called 'bullshit jobs' (which bring food on the table though, but that's another topic).
PS:
> and where software developers carefully will detail stear the work.
...I think the further a project evolves, the less this "detailed stearing" will be any more productive than doing the same without LLMs. The older a project, the more the work shifts from implementation to decision making, and in most cases the result of that decision is just a very tiny code change. I already see cases in my daily work when I use 'agentic workflows' where a tiny change takes longer and involves more 'collatoral updates' then just fixing that one frigging line of code by hand like in the olden days, and for LLM-generated code bases I really do prefer to not mix LLM and manual work, I think that's the worst of all options.
The company that employed the developer ultimately holds the accountability in the marketplace. The employed developer maintains (or loses) their job because of their accountability to their code (or, at least, they should). There's an economic incentive for all parties involved.
In the free/open-source world the incentives aren't economic, but they're still there.
At the start of this you said: "For 99.999% of people, it is literally kthxbye on all code they execute on all their devices."
I think that's inaccurate. The vast majority of code running on "all their devices" is code made by employees of companies being held accountable through traditional industry methods, or free.open source projects where reputational integrity was at stake. Those developers have been held accountable, for some value of accountable.
Maybe there's less value in human accountability than I think there is. Only time will tell. That's a different conversation.
The code running "for 99.999% of people" is not "literally kthxbye" LLM-generated code without someone behind it holding accountability. Maybe it will be in the future, but it's not now.
There is nothing far fetched about this. GNU is an ideological organization and their ideology is "free software." Free software exists because of the copyright system. Now imagine if someone can just fork your GPL licensed software and distribute it under an incompatible license, and when you go to court, the judge tells you to punch sand because your codebase is all AI generated and therefore not copyrighted.
I am not saying this is not what will happen — the actual law seems to be still up in the air. But if it does happen it will be an existential threat to the GNU and the whole free software ecosystem.
> the judge tells you to punch sand because your codebase is all AI generated and therefore not copyrighted.
1. There is no indication that is at all likely except for purely vibe-coded projects. It seems highly unlikely and in some countries (e.g. the UK) the law clearly says otherwise.
2. There have been quite a few rulings in countries where it is unclear, and they all set some level of human input that will make AI generated code covered by copyright. Look at the cases that have been in HN stories about cases in the US, Germany and Japan, for example.
2. It would have to be all AI generated, and you would need to replace all the human written parts. Not a practical problem for a large, old project.
If this is their real reasoning they are jumping at shadows. However, this might be like where, the copyright (which is the explanation given in the ToS) is not the real reason (which was explained in the subsequent blog post).
It is interesting that proprietary software businesses, who have an even stronger interest in ensuring their software is covered by copyright in all countries seem to be quite happy to use LLM generated code. Microsoft and many others boast about how much of their code is now LLM generated.
Without code copyright we are inherently free to modify, improve and share code we have access to. FSF used copyleft as a legal hack to maintain that right in an environment where corporations where using copyright to deny that right.
I don’t think taking copyright off the table harms the practice of sharing code. They will still try to use trade secrets to restrict code sharing and contracts, but using GNU software won’t be stopped. It will reduce the ability to sue people not sharing their modifications but that was always outside the mainstream, and places like AWS, Apple, and Google find ways around it anyways since it doesn’t cover hosted services or non-linked code.
The core stream of openly developed and exponentially improving software does not need copyright to win if it cannot be sued for copyright violation.
Now I suppose some OpenAI lawyer is trying to find a way to sue humans for copyright infringement while keeping them safe from lawsuits, so we can worry about that attack.
I don't think the concerns are far fetched at all. Look at how image models spit out copyrighted stuff all the time. Midjourney has a bizarre EULA clause that if you use it to generate images that violate copyright and _they_ get sued, they can hold you liable downstream. Which is wild to me -- just don't train on things you don't own and this is not a problem!
In images it's much more _obvious_, but I think code is very likely to have similar problems. Like, websites that an LLM spits out are often very very similar. It wouldn't be shocking to me if some of the code in the training set was trained off GPL code, and there are small GPL violations all over the place.
Anthropic and co basically have the opposite policy (for paid users): if you get sued for copyright infringement, they will indemnify you. That means they're confident it's not an issue.
By the way, do you have a source on Midjourney spitting out copyrighted stuff all the time? Does it happen at random or when users intentionally steer the prompt in that direction? I suspect it's the latter but I admit I'm not really familiar with this tool.
These companies have already shown they're extremely reckless with copyright (Anthropic was penalized 1.5B for books, for example). I don't think they've earned that trust of "they looked at it so it must be ok"
The risk might be small -- but is it a risk they need to take at all? We're talking about a very mature project here that more or less needs to move at the speed of standards bodies, which is basically glacial. Being the first to implement C++ 29 probably isn't that big of a deal. People do not upgrade their compilers daily, shit, most projects stay on the same version for years, so optimizing for velocity seems like a bad idea here.
For that very reason they would also add very little LLM code over the next few years (until we get more court rulings) even if they allowed it. A little LLM code would not be usable without the rest of their code which remains covered by copyright.
Its much the same as someone creating a fork of GPL code in which they make additions that they put in the public domain. All the original code and the fork as a whole would remain GPL.
You want to prevent the transition from a GPL codebase with some public domain code to a public domain codebase with some GPL code. One way to do so is to outright ban contributions leveraging tools that are able to generate public domain code at superhuman speeds.
As you pointed out yourself, there's always the option to create a fork that does allow AI contributions, which may eventually force a re-assessment of the policy if the gap in utility grows too large.
> You want to prevent the transition from a GPL codebase with some public domain code to a public domain codebase with some GPL code.
That would take a very long time if contributions are reviewed etc. By then any legal ambiguities would be clear.
> a public domain codebase with some GPL code
which would still be a GPL codebase
> One way to do so is to outright ban contributions leveraging tools that are able to generate public domain code at superhuman speeds.
Can they generate code that would pass the quality standards, and pass the processes, of a project like this at superhuman speed? There is a separate requirement that contributors must be able to understand code and answer questions about it so a human would have to review code before even trying to contribute it.
> As you pointed out yourself, there's always the option to create a fork that does allow AI contributions, which may eventually force a re-assessment of the policy if the gap in utility grows too large.
1. if you are right that LLMs will do well enough to create a huge gap, then that is inevitable.
2. if you are wrong about that then it is unnecessary to try to stop it.
Uhm, removing code once it's in there and load bearing REALLY hard, and breaks a lot of things. You can't just be like "well, I'll just remove the parts with copyright issues". Plus, all those parts with copyright issues you'd basically have to rewrite by hand, which would take much longer than with an LLM. It would be an absolute disaster.
At the same time, they would be somewhat forced to concede those use cases if they wanted to stick to the copyright story. Maybe you're right but I remain skeptical.
Yes but if the concern is being forced to remove code in the case of copyright violation it is trivial to remove code that is a test case as opposed to a feature of the compiler
The stance may be reasonable, but the reasoning is not clear to me and the limit does sesm to suggest copyright concerns that are, as you say, far fetched.
I think definitely a combination; there are those who believe that LLMs are the entire future, and those who are ashamed of themselves for using it. Both lead to rage at people who don't approve totally.
I am making an argument based on the fundamental nature of the tool, which is to make stochastic, unvetted decisions our behalf — categorically different from everything else in our toolbox. The technical and social repercussions to this are potentially vast: no other tool comes close. So it is actually Linus who is making an unconvincing appeal to emotion (and authority) by pretending that LLMs resemble hammers.
I said at face value because there was no reason given in either the policy nor the article submitted here.
It would be interesting to know why they decided for a general prohibition, rather than going with the default "a human must be responsible for the contribution" kind of policy.
Perhaps they have received a flood of undesirable AI generated contributions, and actual contributors do not use AI significantly.
The concern related to copyright is not that the AI labs would assert copyright over the code produced by their LLMs - that is a complete strawman.
The concern instead is that LLMs and all of their outputs may be found to be derivative works of their entire training set, and thus rendered unusable (as the training set is not distirbutable under any license).
I think this ship has long sailed and no court is going to dare give such a decision given the money involved, for better or for worse. But it's a much more realistic scenario, in principle, than LLM labs going mad and attacking their own customers.
Edit to add: there is another, completely different, copyright risk associated with LLMs - and one that is much more realistic. It is the fact that code generated by LLMs may not, in fact, be copyrightable at all. Which would mean that it can't be subject to the GPL. As long as it remains a minority of GCC code, this wouldn't matter much, but it could in time lead to significant portions of GCC becoming public domain, and thus cooyable, modifiable, and redistrubutable without providing the four freedoms.
The mental model of slow decay into copyable is complicated. The code base is already a wash with contributions that have low copyrightability in US case law anyway, things derived directly from external architecture or documentation in barely novel ways, common system interfaces, and so on. LLM output is another output of this kind of class and sure sometimes large contributions happen that seem more significant, but they likely have little bearing on the work as a whole given it's earlier design provenance (at least for established and massive projects like this).
What is likely to get more muddy over time is the accuracy of any copyright registration, and the enforcement of copyright infringements on portions of the whole. These are already complicated cases and definitely so for compilers with so much "scènes à faire".
It's not clear how much this has a negative impact on cases around the whole, which tend to be the more important cases for the four freedoms that, while they have other intentions, have a primary intention of ensuring that the whole continues to be available for redistribution and extension in perpetuity.
I do not think that there is a clear link between these two areas at all, and the GPL's most important intents may be far safer long term than concerns of dilution suggest.
Courts have already set a precedent on this (it's fair use) and even if these decisions were overturned, the liability would fall on the LLM creators for the initial training, not on their users. The outputs would keep being treated individually on their own merits.
I believe there are still ongoing lawsuits on this in various places, I don't believe it's fully settled law - though, as I mentioned in my post, I also don't expect there is any realistic chance for the courts to simply shut down a trillion dollar industry, as such a decision likely would.
I do believe though that, if the LLMs were found to be derivative works of their training set, it would follow almost directly that their output is also a derivative work of that same training set - given how these LLMs operate. And even if the liability fell with the LLM providers (which may not be so clear cut for, say, local models, fine tuning, etc), that would still mean everyone would have to excise any LLM generated content they are distributing.
> were found to be derivative works of their training set, it would follow almost directly that their output is also a derivative work
I doubt so. Let's say Harry Potter is in the training set and you ask the LLM to generate a quick sort function in C, is that quick sort function a derivative of Harry Potter? What if you ask the LLM to output some known public domain work? That leads to a contradiction where according to one definition, the work is public domain and according to the other, it is a derivative of Harry Potter. It seems to me that there's no other option but to consider each output on its own merit.
Not really, no. There have been many settlements of lawsuits, which means they are very explicitly not precedents. The LLM companies likely prefer to settle because of the existential risk being found liable for copyright infringement would result in, whereas even an expensive settlement doesn’t upend their entire business model.
Furthermore, I don’t think you can really assume that the courts will rule a certain way on this just because of the money involved; there’s a lot of money involved when it comes to the copyright holders too, and they’ve long enjoyed a rather favorable status with the courts and legislators. (For example, in the days of P2P file sharing lawsuits and attempts to legislate P2P file sharing, the software industry was already many times the size of the media industry, but the media industry consistently won.)
> Furthermore, I don’t think you can really assume that the courts will rule a certain way on this just because of the money involved; there’s a lot of money involved when it comes to the copyright holders too, and they’ve long enjoyed a rather favorable status with the courts and legislators.
I don't think this is all that plausible, even though I agree with you that it's not settled law. The size of the AI industry is gigantic, and a ruling that they are infringing the copyright of every piece of content in their training set would essentially shut them down entirely. Such a decision, if final, would probably easily wipe out a few hundred billion dollars on the stock market. Even if any court was willing to go that far, almost certainly lawmakers would step in and modify copyright law to prevent this from happening - both in the USA and the EU.
I don't think there is any comparison to make with the file sharing battle. That was a much, much smaller industry, it was not a significant chunk of the total hardware and software industries. Plus, the software titans were not nearly as well connected politically as they are today.
A couple of things: The “AI industry” is nowhere near as large as it pretends to be, most of its valuation is in the form of IOUs being traded back and forth. However, the bubble popping probably will wipe a few hundred billion of value from the stock market, because that’s what happens when a speculative bubble pops.
The second thing is that I’m not necessarily talking about whether _a specific LLM itself_ infringes copyright, but whether _its output_ is covered by the copyright of _its training material_. Whether training an LLM is an activity that infringes copyright is not well-settled in any precedential way, whether the trained LLM as an artifact infringes copyright is even less settled, and whether the output of that LLM is either infringing or covered by copyright is also not settled. These are all still extremely open questions.
That means anyone doing reasonable risk management should not just blithely race ahead and assume that there’s no infringement, which appears to be the approach the GCC project is taking explicitly and which also appears to be the approach projects like Linux and LLVM are taking implicitly (mostly through weasel-language like accepting responsibility for code you’re submitting).
"To summarize the analysis that now follows, the use of the books at issue to train Claude and its precursors was exceedingly transformative and was a fair use under Section 107 of the Copyright Act. And, the digitization of the books purchased in print form by Anthropic was also a fair use but not for the same reason as applies to the training copies. Instead, it was a fair use because all Anthropic did was replace the print copies it had purchased for its central library with more convenient space-saving and searchable digital copies for its central library — without adding new copies, creating new works, or redistributing existing copies."
IANAL and don't know how significant this decision is, but it is, at the very least, how one judge views it.
Personally, I don't think judges will rule a certain way because of the money involved but because it seems clear that training a ML model is highly transformative.
Not quite the precedent that it may sound like. A district court judge ruled that using copyrighted materials for the training itself was not infringement, but that the materials must be obtained legally.
Anthropic is trying to settle the case with most plaintiffs with respect to obtaining their works in an infringing way, but there are still plaintiffs pursuing the case on both the grounds that the remedy is insufficient (being only about $3000/work, when it has been as high as $250K/work in other copyright infringement cases and via statutory damages) and also on the grounds that the ruling that training is fair use was an error of law on the district court judge’s part.
Notably it doesn’t cover whether the output of the trained LLM continues to attach the training set’s copyright, which is independent from whether the training itself was an infringing activity. And there’s a substantial argument that the judge erred, if it can be shown that the training works are stored in a recoverable manner (even with some loss/defredation) rather than more extensively transformed.
In many timelines, non-copyrightable code in GPL project doesn’t restrict the user’s freedom, since an LLM can easily reimplement the functionality clean-room. AI has brought levels of freedom to users that GNU could never do. Practical levels of freedom, leading to real, positive outcomes. Software is finally free in the real world.
For certain use cases, probably. For others, no. For example, hardware that uses GNU software must be patchable by users (if it is patchable at all). Hardware running public software can be completely locked down to its users, regardless of how powerful of an LLM the user has access to.
> Someone re-licenses GCC under a non-GPL license.
A pointless act since code is now free. The GPL exists to ensure code freedom in an era when code was expensive. Yes I'm aware that the meaning of "free" is a bit conflated here, but the point stands.
I believe you are underestimating the value of GCC and LLVM having won the compiler wars and being extremely high quality, and of Linux having won the OS wars, being extremely high quality. I use GenAI every day, but the quality is still proportional to the amount of my time - there is a multiplier for quantity of my amount but the 9s approaching 1 quality is not logarithmically different - the effort to fix one big is lower, but the total effort to make excellent software is still a process of polishing and then iterating. The embedded quality in GCC will take a similar order of magnitude investment from an LLM rewrite. These super high quality open foundations make LLMs more useful to us now, but saying that the embedded value or cost of GCC or Linux is zero I think will steer people wrong.
IP law (like a lot of other things) has been skewed toward the interests of business, even when that conflicts with fairness or societal good. For all its flaws (IMO), the free software movement tends to be principled. Just because something is legal doesn’t mean it’s right.
Not just when it reproduces code. Since fair use doesn't exist like that in a lot of countries the question is if the training itself is a violation. Are there e.g. EU or Japanese rulings on that?
> To my knowledge there is not a single known case where this happened.
> News back then were about intentionally prompting to output known copyrighted material.
First, there are other cases if you take the time to dig. This is quite an old example (GPT-2) as i haven't kept up to date on this field recently, but it does show that this problem has been known about since before these systems were widely adopted: https://arxiv.org/abs/2012.07805 [0]
Second, GP said nothing about the type of effort required to make it happen, just that it can be done and that the copyright owner could come along and cause legal problems later. It's absolutely possible to have a fly-by contributor who purposefully asks for code that reproduces X/Y/Z without a maintainer knowing about it.
But then the maintainer is the one in legal trouble.
> When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest?
Legal / copyright / etc. cases often take a lot longer than a couple of years to come to fruition.
---
[0]: edit -- to clarify this is an example of the reproduction problem, not an example copyright infringement case.
The problem you linked is an older example of intentional prompting for copyrighted material.
The concern discussed here is copyrighted material being generated unintentionally and the original author asserting their rights.
This has, to my knowledge, not happened once.
If we are not talking about unintentional violations, I don't understand the point of the discussion.
I can also intentionally copy paste the copyrighted material into my merge request without the use of AI in an attempt to get the maintainer into trouble.
> The concern discussed here is copyrighted material being generated unintentionally and the original author asserting their rights.
both intentional (malicious contributor) or unintentional (Large-Laundering-Model) are copyright issues -- which is the point of GCC's policy.
> I can also intentionally copy paste the copyrighted material into my merge request without the use of AI in an attempt to get the maintainer into trouble.
You can. You can also do it significantly faster with significantly less effort while being harder to detect using agents etc.
I do not think some untested theory about sabotaging open-source projects by intentionally inserting copyrighted material using AI, causing legal issues for maintainers, is worthwhile to discuss here.
That is obviously not what anyone was referring to, nor does it make sense, when there is a much more reasonable basis to prohibit the same contribution.
Namely inserting vulnerabilities. This one actually happened before afaik, and provides a clear benefit to the attacker.
> just that it can be done and that the copyright owner could come along and cause legal problems later. It's absolutely possible to have a fly-by contributor who purposefully asks for code that reproduces X/Y/Z without a maintainer knowing about it.
If a contributor doesn't care about submitting copyrighted code, they can do it without an LLM as well.
Someone tried to contribute “vibe-coded” device support to a project I’m involved with, they said they did it all based on the device documentation, the code their agents spit out was copied verbatim out of a (GPL’d) project with which I’m familiar which supports that device.
LLMs are not learning things and then using that learning to construct new things. They are essentially a form of lossy compression of their training set. And you don’t need to be explicit about trying to reproduce a portion of that training set for an LLM to output one.
As it happens, all evaluations I have seen in the news were in fact explicit about trying to reproduce a portion of the training set.
I am not aware of any study attempting to measure unintentional reproduction.
With your example, I question whether you have seen this happen first hand. For all I know, the contributor could have explicitly prompted the model to reference the GPL project and had the agent clone the code from the web.
At a certain point you have to take people at their word; I’m reporting what the contributor said they did (used the documentation to generate the code).
I mean ai has generated copyright headers verbatim in code I asked it to make. I had a tutorial open from the web and asked the AI to generate an example for me and it was the example from the tutorial verbatim as well...
It was a frontier model from a few months ago. It hasn't done it recently. But it has happened. I imagine if there isn't a lot of training data for an idea it probably has a higher change of generating verbatim, so as time goes on, it should become less of a problem (at least with high quality models).
This doesn't really happen by accident, especially where it's significant enough to constitute copyright infringement. And to my knowledge, there hasn't been a single lawsuit over this in the entire history of LLM-assisted coding.
It's enough to tell you that, if it does happen, it must be quite rare and not a widespread systemic issue. It's also worth noting that Anthropic and co are so confident that it's a non-issue that they offer full copyright indemnification to their commercial users.
That would be an example of, “overfitting.” It is something the AI labs are concerned about. They tweak parameters to try and avoid it.
It’s a hard balancing act to do. Give in too much randomness and you get non-sensical outputs that are difficult to align. Fit too closely to the training data and the model regurgitates the training data.
And oh, what’s that copyrighted material we never made any agreement to use doing in there?
There is concern, but the more I think about it, the less convinced I am. LLMs might produce a snippet, but copyright requires more than that. Last I checked (I'm not a lawyer, this was far enough back that the law has had major changes, and different countries have different laws! - read this with plenty of salt) if you start with copyright work and replace all but 10% that final amount just disappears since there isn't enough original to claim copyright anymore. You can never claim copyright over 'printf("Hello World");' - it is too generic. Most of what I've seen an LLM produce is likewise generic enough that even if it is a copy, it wouldn't be a copyrightable snippet.
> The complaint argued that "the basis of the Gaye defendants' claims is that "Blurred Lines" and "Got To Give It Up" "feel" or "sound" the same. Being reminiscent of a "sound" is not copyright infringement. The intent in producing "Blurred Lines" was to evoke an era. In reality, the Gaye defendants are claiming ownership of an entire genre, as opposed to a specific work"
wider point -- whether or not a copy is a copy and whether it is is infringing on copyright or not ultimately has to be decided by a court case when it's not an obvious and clear cut violation. especially in the USA with the utterly mental fair use law.
He's correct but it's an irrelevant argument, he's simply making an emotional, and very childish, attack on someone because they don't like the tech he likes. Banning LLMs in your project, regardless of the reason, does not mean you think they are ever going to go away, or even that you want them to.
You think that not a single powerful country in the world will get enough pissed-off people in power to successfully sue LLM labs for (blatant and willful) copyright infringement? Especially if they succeed in decimating the job market?
How do you build a legally-sound product using an LLM that has been successfully sued for violating copyright in X countries around the world?
I believe there’s a small minority of LLM users who will show up to any comment thread and scream bloody murder if you post any kind of policy limiting LLM use. They seem to feel like it’s a personal attack and must justify their use of LLMs… for absolutely no reason. It’s wild.
Yep, same for those LLM creators who get all pissy when you provide (constructive) criticism about what they've outputted (or ask reasonable questions like if the person is doing actual tests) and they get all offended and call you ungrateful. This happened to me with the author of a Godot GDExtension: I'm (pretty sure) the dev was vibe-coding it, so after like 4-5 releases of it not loading in the editor at all due to what I would consider basic mistakes, I bluntly asked them if they were actually testing their code and got told I was ungrateful and being rude and disrespectful and all that even though what I was asking was very reasonable (after all, the dev had repeatedly released versions that wouldn't even load in the editor).
It's like a gun enthusiast throwing a fit because you don't want them to bring their gun into your house. "I get that you find it useful to have a gun at your house, and I understand that we are never going to revert back to the days when guns didn't exist, but we don't allow them in this house."
(and not, I'm not slyly trying to say LLMs are like guns, yeesh)
It feels like we're witness to the possible appearance of a new archetype: a whole new kind of capital-P-Person! A hitherto-unnoticed plane in personality space is slowly becoming discernible.
(Time will tell. It's possible this will turn out to be just one of the same old People that you've already met, with just some minor tweaks to the specific details...)
Last time I interacted with people who push these "policies", I saw my projects be dismissed as slop because LLMs had touched them, was called a "slop fetishist" in public and was told to leave in no uncertain terms.
That guy is stupid, the part of human nature containing dishonesty also houses "following the path of least resistance", which means that these people will simply contribute to another leading compiler without these rules.
Full text is linked below, but it is getting revised pretty regularly.
tl;dr: Requirement is that a human must be in the loop; the contributor must have reviewed the change by hand already; and is always accountable; the human must be able to answer questions about the change, such as strategy chosen, corner cases, etc. etc.
Even with this very permissive and well considered policy, they get a lot of slop submissions. Huge amounts of pure trash. And the debate continues about what to do about it.
Some are in favor of forbidding it simply because it would reduce the amount of slop they have to wade through, and the 10% good prs done with AI don't outweigh the 90% pure junk prs. Reviewer time is way too scarce.
But it is just as controversial there as it is over on lwn.
Side note, but playing Talos Principle after the AI revolution hits different than it did back in 2014. So much of what Milton says feels a lot less hypothetical today.
It is on par with Portal 2, both the original and the sequel, imo. Some people think the sequel took a step back but I don’t really see why, if anything I liked 2 better than 1.
Maybe it's not a full and clean correlation, but there still is one. In my experience, AI boosterism is often linked to conservatism, because the current US administration is all in on AI, the spoils of AI disproportionately reward the rich and increase inequality, and because leaving AI labs alone is fully consistent with most conservatives not wanting to bother private business and letting them do what they want.
The thing with AI is there are multiple axes of division that are in tension, and they don’t map conveniently to standard politics.
For example, Anthropic in one sense seems to occupy a caricature of the nanny state worldview, with their constant calls for safety regulation. But then when you examine the motives, it becomes pretty clear that giving them what they ask for will give them unprecedented consolidation of economic power. So if you are a liberal normally inclined toward regulation, you have to consider: is it worth living in a world with even more trillionaires controlling an even larger share of the world’s resources? Or would it be better for AI to become fully commoditized so that its benefits are more broadly dispersed?
Or suppose you are right wing, free market capitalist, deeply nationalist person. Are you for or against Chinese AI models? Your desire for America to “win” AI may be tempered by knowing that the biggest winners will be some power-mad EA people in the SF Bay.
Judging from social media interactions, this space seems to make for strange bedfellows at times.
My counterpoint to that first one is that forcing outcomes through laws doesn't have a political side, although the discourse in the US has definitely been extremely mangled by the untrue stereotype that conservatives are less keen on passing legislation. Strong government force can be used to either pass real regulation or to create and protect monopolies. Anthropic is asking for the latter. None of the AI labs would ever willingly ask for the lid to be put on their pot, it's just everyone else. Their demands are just requests for a government-protected monopoly that are PR-worded to appear as regulation. Left-leaning people are usually proponents of real regulation, the kind that businesses would never ask to be imposed on them. Right-leaning people may side with the AI labs more, seeing their proposals as compromise or reading between the lines and supporting the regulation to ensure that their side wins and that the biggest labs are handily rewarded for their work.
The Chinese model split is more interesting, but only as a theoretical point that examines what different political sides would support in theory if everyone's ideology was fully consistent with itself. If you look at the people, in reality most conservatives seem to side with laws that would protect their own and ban other models to 'win'. Left-leaning people are more likely to be okay with Chinese models or open-source AI, seeing them as opportunities to dislodge the power of American AI labs and prevent too much power from concentrating in few hands.
So I think the correlation is still valid. There are a few people on all sides who may take an unexpected worldview in light of these new problems, but I think that for most people, what I outlined is more or less the way they've split up.
There has never been a time when open source isn't full of flame wars of one flavor or the other. Are we going to diagnose people with syetemd psychosis or X11 psychosis next?
It is reasonable to stop some people from flooding projects wirh low quality PRs that take time on the other side, that might not work at all or be patches that are several times the size of what is needed and get a lot in the way of the maintenance.
I agree with a policy of "no AI contributions by default" just to be able to ban them quickly and lower the incentive.
Ido not have anything against AI itself though, as long as it is lanaged by humans and snippets are properly reviewed. But that is not what many ppl do.
They will just drop something there and say: you silly, review for me amd I take a lot of credit.
I would not spend a minute in that kind of contributions.
For me I’m a late adopter. I’ve seen more things go than stay. I’ll wait until the industry has stabilised or evaporated before making a decision. It’ll save me time and money.
Because discourse around AI has devolved into eschatological thinking. The stakes are potentially high where the ceiling might be superintelligence and the floor total economic collapse, humans will just default to apocalyptic thinking. All discussion ceases to be technical and instead becomes religious.
Once people have convinced themselves that the stakes are existential, any nuance or moderate thinking feels like complicity.
So now because of that, a perfectly reasonable policy about copyright gets reframed as a battle for the future of humanity becausae we are no longer discussing tech, we are discussing what amounts to opposing religious beliefs.
I don't know, it feels like the GCC steering committee's AI policy is sorta the definition of moderate speech.
They aren't coming out swinging on LLMs shouting it down as slop and calling anyone using it lazy. They just created some, reasonable to me, guidelines that about when the use is and is not allowed in their project.
