Here is a list of open-source projects that reject AI-generated code.
The Languages section includes Idris for example.
Is there any plan or any hope to have there as well:
JF
I believe OCaml (as pointed out by a project a lot similar in spirit to the referenced project) has somewhat of a lenient “merit-based” LLM policy. with @dra27 for example, a core maintainer of the language, using LLMs occasionally on a couple of non-core pieces of the OCaml distribution like the tests.
Janestreet-adjacent or -sponsored software like oxcaml and dune has fully embraced it. odoc and the vscode platform somewhat as well. sedlex but not utop. So far OPAM, ocamlfind and Merlin seem to be untainted from the blessed core set of OCaml tooling (ocamlfind makes sense since it’s what I’d call “complete”). @dbuenzli 's software also counts here as “core tooling” IMO as it offers viable alternatives to some of the tools I mentioned. I am unaware if Daniel made a positive/negative statement on the possibility of LLM code being in that software. But that’s probably me not keeping up.
All that is to say, while I deeply wish OCaml was on this list, realistically that doesn’t seem to be the path upstream is taking.
I’m not aware of a venue to influence that path, and I make no comments on whether one should exist if it doesn’t.
For the time being I have no plans to use LLMs to write the code I distribute (for many different reasons, not worth expanding on them here though, I think we are all pretty tired about these discussions). You should take the lack of AGENTS.md in my repos as a signifier.
Now if people show up with PRs that were crafted with the help of an LLM I will judge on a case by case basis. But my first gut feeling is no (see here for example).
I just noticed yesterday that the primary author of Bogue does not use LLMs for development, in furtherance of a low energy consumption principle.
Personally I am unsure on this topic. I don’t use LLMs myself for programming, but I don’t feel that I understand the topic enough to judge other people who do, and that my judgment would stand the test of time. So I cannot strongly support policies that forbid their usage entirely, I would like each contributor to make their own value judgment on which tools they use to program. (I’m sure that some people will make choices that I disagree with, but I feel that it is safer to allow this to be able to work together as a community.)
On the other hand, I do care about quality so I feel very motivated to discourage or forbid practices that hurt quality. Letting people make choices about their tools does not mean that I accept bad outcomes or that I am volunteering to waste time reading slop – let’s have policies to prevent that. But I prefer policies about the actual quality of the outcome (paired with a requirement to be transparent about tool use, which I think can limit abuse).
I also think that “completely LLM-free” is not a good metric for large free software projects that are open to external contributions. If we were interested in keeping this badge of honor, we would have to put a formal ban on AI usage and severely restrict or police external contributions to hunt people who cheat. Policing external contributions to guess AI usage is exhausting and a negative thing to do and I don’t want it – realistically the only way to avoid burnout is to stop accepting PRs for the project, and indeed some projects are going that route.
I feel that any more permissive approach to regulate AI usage in large projects is inevitably going to end up with some AI-assisted contributions (because more people are using them over time, random people contribute to the project, so inevitably this will happen or has already happened), so the proportion of projects with some AI will become 1 fairly quickly, outside niche communities. I would be more interested in more nuanced questions about AI usage in a project, for example: which portion of contributions are AI-assisted? Are all changes human-reviewed, or what’s the proportion of AI-only reviews or no-reviews? What steps (if any) has the project taken to preserve quality after the change in programming practices that happened (if any)?
I also perfectly understand that some people have an extremely negative perception of the externalities of LLM usage. In general AI tools are uniquely negative in various ways, compared to most other programming tools: they tend to rely on underpaid human labor, the current crop ignores copyright, the energy usage of IT is sky-rocketing, and the current trend is in the concentration of value in the hands of a few large companies (maybe this will improve with the slower-but-steady progress of local models). I understand why some people consider that all these costs justify a blanket ban on AI tools for programming. I personally don’t feel confident enough that I understand the costs and the benefits to justify a blanket ban on the projects where I would be legitimate to push for it.
This is a bit of a tricky point because “we don’t know the costs well enough to make a harsh decision” is also a very easy cop-out for the industry. Big players lie about energy usage, distract us from the labor or copyright issues, etc. And people who grew addicted to these tools can conveniently forget about those costs to avoid cognitive dissonance. (Many statements on LLMs these days read like “I know that there are ethical issues but {rest of the post that ignores them}.”). So personally I am glad that there are people who are acting on these ethical issues, trying harder to find out how to measure these costs to have a real conversation about it rather than looking the other way. And I suspect that the movement to reject/ban LLM from certain communities is making these questions more visible and giving energy to this crowd, which I see as a positive.
I’m generally inclined not to want to use AI, and I find (even if this isn’t an exhaustive list) that the reasons you cite here particularly resonate with the criticisms I have regarding AI; the issue of copyright is one I’ve only just (belatedly) realised, and which was, unfortunately, the same for artists.
The environmental issue is the second criticism that springs to mind.
What I find rather difficult, unfortunately, as someone who maintains software, is not only the amount of code that can come our way and that we have to review, but also the lack of honesty on the part of some contributors who fail to disclose that they have used AI, as well as their unwillingness to even respect the time we spend on the review (I’ve already had to deal with a complete rebase of a pull request during my review: so I had to start the whole review from scratch, which was very frustrating).
Finally, it also feels as though some people think our job consists solely of fixing bugs or adding features, which often leads them to completely reimagine everything they want, forgetting that building a community and/or an ecosystem necessarily involves discussion and collaboration with others (and, of course, that takes time).
PS: I do still use AI for translation, as I’m a long-standing user (long before AI tools appeared) of DeepL.
Just to add about my experience in our company, where one division completely jumped on AI assisted code generation bandwagon. The result is a complete mess of apparently working code which passes particular tests, but it introduced several later discovered serious security issues, not experienced years ago with hand-crafted code and mostly, devs no more understand code they deliver. It is now forced to work around this issue with many additional tests written in a classical way, but the question remains how sustainable will be this approach in the long run.
(accidentally ended up writing a lot, sorry! TL;DR prevention and policing is the wrong thing to emphasise IMO. Larger projects already addressed this point. Bad actors cheating the policy may be overblown. Lenience may not be a good solution. Nothing of the social dynamics changed between this and other contribution boundaries. It’s ultimately a matter of priority but I think concerns should be fully and adequately addressed no matter the choice made)
It might be worth checking how open projects larger than OCaml (by number of contributors and volume of submitted work: qemu, openjdk, zig, gentoo, …, even arxiv and wikipedia) addressed this point. Opening a communication channel with them over these concerns, or reading their publicly available discussions.
I suspect not many are putting policing at the front with their policies. Example.
Rather, it’s good to remind ourselves that contribution to free software is polite and voluntary. A strong stance (against or with) would still be part of that polite and voluntary agreement.
It’s strange to my eyes the weight given to anticipating a proliferation of bad actors specifically in this case, and potentially deciding it’s impossible to take a stance, even though we might be interested in it, for that hypothetical. We already have examples, we can ask them how much of a problem this actually is.
Just like it’s always been expected from contributors to correctly attribute work and respect licensing; despite the logistics of detecting and preventing bad-acting plagiarizers being difficult, sometimes impossible if the plagiarized work is proprietary; with maintainers not being expected to have the bandwidth to investigate every contribution for such violations, only to respond appropriately when an issue is raised… Just like it would be with this.
That is to say, I think “hunting those who cheat” isn’t the best to focus on.
There are reasons big projects, ours included, don’t go the easy & most practical route, instead putting more deliberate work in articulating their boundaries and in fostering community engagement through foundations and boards; practicality shouldn’t be the only concern, communities are complex organisms.
Still, practically, it might be more work to be friendly to LLM contributions and thus be forced to review them on their merit, maybe even go back-and-forth on that vague measurement of merit, when the effort put in them is much lower than the produced output needing review… This will eventually constrict maintainers’ tolerance to less-than-perfect contributions (rightfully so, to avoid burnout) which is unfair to people who are putting actual effort in learning the codebase and trying to improve it.
That or plead to yet another probabilistic review tool to Do The Right Thing, in order to handle the volume.
It shouldn’t be easy to forget how hostile to the community’s established efforts unrestrained LLM output can be (ocamlwiki), nor how spread-thin the reviewers were before LLMs came into fashion.
So I’d say it’s less prevention and more “messaging” and “drawing a boundary”…
A calculus of “is taking a stance on the aforementioned ‘externalities’ worth potentially alienating prospective / known contributors who are on the bandwagon and refuse to respect this boundary”. Such calculus is at the heart of community stewards’ role I think, and defers to / reflects the community’s values. Enforcement, if any, is a social not a technical question.
I said in a different thread: cheating the policy requires more work on the contributor’s side to be convincing, and requires them risking their social credit in that project. It really is more about whether you want contributions from people who have less incentive to contribute to the project in general than to specifically point a text extruder at the maintainers.
I want to also add: there’s no denying efforts like open-slopware are somewhat uncharitable to software projects and do play on “honor and shame”. Though I believe such provocation has its place in the generally ethics-poor practice of software. I think our community does better job than many.
“Do we want to welcome openly bigoted contributors”, “do we want to be sponsored by weapons manufacturers and host them in our conferences”, “if we’re capable, why aren’t we offering protections and opportunities to structurally disadvantaged community members”, “who are we locking out of the contribution process by requiring xyz”, these are provocative questions that might be distasteful to, might even be taken defensively by, those enjoying the appearance of peace and harmony, but IMO they ultimately result in more resilient and independent volunteer projects.
Industry and the volunteer-maintained digital infrastructure. You’d think the latter would be less beholden to the brutality of market forces and thus less incentivised to follow harmful trends.
open-slopware and its likes do a good job of collecting points of concern—fashionably ‘externalities’, though I insist some parts of them are inherent & internal to the technology in its current form and thus cannot be decoupled from it—for digital infrastructure projects which choose a more lenient or even positive policy to try and adequately address (or shut down, that too is a position, the question of how responsible it is I leave aside).
Thanks! That’s very important to point out: the overload of “AI” is also a marketing strategy and we should be more precise with our terms in our non-marketing discussions.
Translators, TTS and STT, OCR, scientific models… There’s a wealth of algorithms which fall under the “AI” umbrella but do not admit most of the problems mentioned here (Of course they have their own problems with systemic biases, worker exploitation and unconsensual/unlawful data collection etc. Like I said, software practice tends to be ethics-poor).
Discussing them as an example of “good application of AI” is a distraction which the recent probabilistic text generators and the industry around them benefit from.
To be clear: if someone sends a PR that is bad slop, we already have the tools in place in our contribution guidelines to reject it. (And even before we formalized anything, we already did reject it.). For example the last point of AI.md states:
The following are never acceptable as contributions: low-effort productions of code that contributors do not understand; plagiarised works; hallucinated security reports; or experiments involving humans without their advance consent.
The problem, to me, is with the contributions that used AI tools in some way, but where the submitter took ownership and the result is of good quality. For example, maybe they used AI to generate preliminary version of a unit test for their PR, and then they read and simplified/improved the test, and the result is good.
Some people feel that AI tools are great, improve our productivity, and that virtually everyone will use them in the future (because, presumably, they will become excellent and cheap and local and worker-owned?). Some people feel that AI usage is bad enough that even “high quality” use of it should be banned. Personally I am not sure. I have enough opinions about them to decide on my own usage policies, but not confident-enough opinions to force my esteemed colleagues to not use them, when I cannot articulate from the output of their work why it is a bad thing.
Regarding policing: I think that my experience emphasizes policing more than you do, in part due to my part-time role as a teacher that has to define what is considered cheating, and then enforcing those rules. (This field has been disrupted before open-source projects because students started using LLMs before agentic coding started producing decent-looking output.)
I spent some time thinking about the following question: why do some students that are clever, well-meaning, respectful of their teachers in general, would still ignore my “no AI tools” policy in their take-home project? My conclusion is that some students absolutely do not see the problem with AI usage (despite the fact that their shortcomings are even more obvious in a learning context). When I ask them to not use AI tools, some react as if someone asked me to not use an editor that supports automatic indentation, or syntax highlighting. They don’t understand the rule, so it doesn’t click, and they don’t apply it. When they don’t follow this rule, they do not feel or act like bad actors; they don’t realize, or they think that I am slightly deranged and that this aspect of my requirements can be ignored by any sane person.
I think another theory is that its an easy way out, and poor planning/organization can make students pretty desperate with regards to take home assignments, so even if the rule is understood, when sufficiently close to the deadline it seems like the alternative is worse.
I think that this is different (but maybe this is off-topic so this will be my last post on this specific part of the discussion). You explain why people feel forced to do something that they know is dishonest. I am trying to explain why using certain classes of tools genuinely does not feel dishonest to some students – and to some programmers – to the point that they may ignore (voluntarily or not) rules about them.
I don’t want to join this debate (though I have -strong- feelings about the subject) b/c I’m not a maintainer of OCaml. But I appreciate that the maintainers are having this discussion: it’s important and something that will affect the health of OCaml going forward.
I browsed some of the links given above and ended up on the OpenJDK AI policy. I like it, especially since it states clearly that the output of generative AI tools create copyright hazards (my impression was that the whole industry put its head in the sand on this topic, and it is sort of interesting to see Oracle being careful about it here).
It also ends with the sort of advice on detecting AI contributions that I personally find difficult to apply and painful, in terms of reviewer morale, to consider. It starts in a somewhat expected way:
[As a Reviewer] you are already expected to do your best to ensure that incoming contributions are consistent with OpenJDK Community policies and conventions. In general, reliably distinguishing human-generated content from AI-generated content is impossible. If, however, you see evidence that content in a contribution was created with a generative AI tool, then it is your responsibility to notify the contributor of that fact.
… but the end is kind of scary to me
Other clues include highly structured comments with multiple headings, unnecessary comments in code, gratuitously defensive programming, and the use of emoji characters. […] In general, if something in a pull request seems uncannily cheerful or meticulous then you could be looking at AI-generated content.
In general I don’t want to have to train myself to recognize AI-generated code, because I believe it can easily make people miserable.
( @hyphenrf had some good points on the fact that this may not be necessary if the policy is communicated clearly enough and the goal is to set a general direction, not a strict enforcement, so that assuming good faith on the contributor side is enough to derive value from these policies. )
Yeah I understand, and all the chances of dialogue I’ve taken carry no implications beyond voicing my mind, keep bringing up the issues I care about hoping to see them addressed by the community & leadership, hear & discuss POVs etc. It can be tiring, everyone has an opinion on it… But that’s because it’s contentious and the implications for harm that takes lots of future effort to undo are high.
Certainly not me hoping one stance or another is forced onto the respected maintainers & contributors. I believe it’s not my place:
I’m not aware of a venue to influence that path, and I make no comments on whether one should exist if it doesn’t
And I recognize there has already been broad maintainer consensus on the status quo as mentioned on GH. My loud sentiments might not even reflect the larger community’s values. Thankfully there are ways to participate and objectively measure the silent majority to a degree, at least as a point of reference.
–
“No bad slop” is a good technical barrier. IIRC Linux was cited in the initial GH discussions for OCaml’s policy. The choice has famously been made by the respected Linux leadership to refuse to address non-technical concerns. “Please don’t bring these up” does count as a stance unfortunately.
For the aforementioned projects, as I understand the rejection would be for what is technically and non-technically implied by accepting the method of production in artifacts they maintain. And I don’t think they attempt to track what people do with their process outside of that production, which falls onto personal ethics and responsibility.
Since the output is a collective responsibility to cultivate and maintain and many participate in its dev process, it falls under the project’s agreed upon boundaries, not the individual contributor’s values.
I agree there would be little argument to reject a plausible output based only on technical merit at this point. Though if nothing else, it doesn’t sit well with me to depend on a subscription service maintained with the compute power of several bitcoin mines to consistently get that “good output”.
Actually imagine something ridiculous like, GH allowing maintainers to enable a system where every PR review mints a NFT ticket you’d use to do blockchain gambling, relying on the human tendency to gamble. On purely technical grounds it might end up offloading a lot of maintainer work and bringing in high-quality reviewers. Is that enough though? Surely there are better ways..
Same idea in my mind.
Is this a set of values the maintainers want to think about or officially open non-personal dialogue for with the wider community? I don’t know.
–
I can’t say much, as I’ve never been one. I have only been a student and a helping peer not responsible for whom I teach.
From that POV: there are those who do understand how harmful cheatron3000 is to the learning process, that they’re doing themselves a disservice by not engaging with the tasks you give them. Students respect you as a teacher and value your work, but still might find the institution’s requirements—carried through you—too rigid, arbitrary in amount, transactional… They have decided the fulfillment of course requirements (or even the whole degree) is more important than fully learning the course material. That’s one practicality I’ve seen many students voice in various ways.
So structure, purpose, volume, who gets to decide topics and fulfillment criteria, those all typically play a role in this dynamic.
And that is one way the educators-student dynamic is different from the maintainers-contributor: the work is imposed onto the student from a position of authority to fulfill certain requirements. And failure to fulfill them can be very costly.
A typical contributor volunteers their free time and labor on their own terms. No-one is giving their work a letter grade. They can re-attempt it as much as they want. And they want to do this task, they might have even come up with the task on their own. They want to improve the software they use for themself and others.
The incentives seem very different, and I postulate so would be the incentive to self-justify the use of these services or bypass a ban on them with dishonesty, between the two groups.
–
If the forum would allow me to put more of myself here
It confuses and frustrates me as a young starry-eyed programmer who looks up to the old guard, the mentors and elders, the role-models of this craft, how many are enthusiastically doing themselves and us new gens a disservice; so easily abandoning in droves the constitutional knowledge they dedicated their lives to… Driving forward the ruin of what allowed us to take after their experience, the volatility of work, muddying of knowledge, the artifacts of their design and thinking process, lost in closed ephemeral non-deterministic feedback loops, all for some supposed future “inevitability” where engineering highly precise and reliable specifications for humans to read and computers to execute is no longer needed. Really..?
Free Software is where I expect craft to live and be passed down when The Market mandates deskilling and automation. What is worth them capitulating to companies which have vested interest in enclosing the very commons they worked tirelessly to maintain?
Yet most of the negative sentiment I’m seeing from my peers not those at the height of their career.
Why is a probabilistic algorithm more worthy of time writing detailed hand-holding documentation to it, hoping to reduce its failure modes, than the humans working on the software?
On a more personal level, why would they so easily hand over the child of their labor and the product of their love to be reshaped beyond recognition by some high output black box?
And it is with such cynicism and insistence that there is nothing to do about it, that it is foolish to push back on it or discuss how to achieve the utility without accepting the current form with all the harm it brings. It feels awful and dreadful to witness.
On a purely philosophical perspective the technology itself is fine and can be developed in a way which is beneficial to everyone. It is fine for programming to be abstracted. We have abstracted for as long as the practice existed. What’s problematic is the technology form today. It is NOT and never will be in its current form that platonic ideal many are chasing / anticipating, and subsequently accepting active harm today as a price for it.
Yeah I think it can be a source of toxicity to go about it by doing rigid detection & vigilance. I’m firmly on the statement of intent camp. But copyright issues are difficult to address from that approach. I don’t know.
Echoing some interesting thoughts here:
the question of enforcement of an LLM ban can be contentious […] in particular, i believe it’s ridiculously important to avoid the misconception that LLM output can be identified by looking at it.
[…]
enforcement of LLM bans can’t whatsoever expect to identify LLM output, and enabling accusations of LLM usage enables toxicity and even xenophobia.
It’s by no means reliable or desirable to codify and have reviewers act like LLM police…
A good reason for some project to take a stricter stance is that some project have enough prestige that a lot of people want to be a contributor. I think the linux kernel is a good example. Because of LLMs, writing code becomes a lot easier, and reviewing is just as hard, so even if the quality of the PRs is okay, there can be a big issue there.
Hopefully for a smaller community like OCaml, most contributions will be made in good faith in the sense that the person opening the PR does so because they want the feature or fix to be in the language or tool, not for some external reason.
I’m just an onlooker (or an on-reader I guess) in this discussion, but I wanted to leave an appreciative note for the participants in this topic, because I think you have all brought up thoughtful perspectives and I appreciate the respectful tone (people admitting that other perspectives are reasonable/being sympathetic to each other and so on).
Makes me happy that this is a community I associate myself with.
In addition to usual arguments against AI-generated code (environmental, ethical, …) which are sufficient for me to ignore and not use these codes, there is also a practical argument: when the author of a code used an AI to generate it, this is (often ? always ?) the sign that the author did not take enough time to get a good understanding of the problem to solve / the domain of application / the protocol to implement / the library to bind to / … and this can reveal a lack of interest and no middle or long-term maintenance. Depending on such a code may not be a good choice.
So, could we add an information in opam package descriptions to indicate when a package was developed using AI and add a way for the user to indicate that opam should not install such packages ? (or park these AI-generated packages in a separated opam repo so that the user can add it to its opam repositories or not)
I want to share my perspective, which is from someone new in the OCaml community. I’m familiar with OCaml since 2010, but only recently I got back to it. In fact, this is my very first post in the forum. I joined this community because I wanted to share about a library I’ve been working on, but I felt a bit discouraged to do so after reading this thread.
The library I’m working on is reativa, it’s a library for building web UI interfaces with OCaml and ReasonML using a js backend like Melange or js_of_ocaml. It was 100% AI generated, ported from another library I created a while ago, xote for ReScript, in which I also used AI tools to assist during its research and development. Despite the AI usage, there was a lot of intentional planning, design and thinking behind.
While I agree there’s a problem to solve when considering AI and open source contributions, I don’t think discriminating projects or even packages fully written by AI would be a good direction for the benefit of the community and the language ecosystem. Slop already existed in open source world before AI, it’s not something new. It did scale the problem, but we should be able to come up with better systems to address it. OCaml having a niche community compared to other ecosystems is an advantage from this perspective, which is something already raised in this thread. You shouldn’t add a dependency to your project, unless your understand its value. This didn’t change with AI.
Related: rust-lang/rust is adopting an LLM policy | Inside Rust Blog -
Some parts of this do a good job of articulating why the current change in the modes of production calls for response in explicit policy.