ah, i see the issue. The warning in the repo file failed to take into account beta-testers of opam 2.2.0 since the builtin warning exists only since either 2.1.6 or 2.2.0~beta2.
I’ve opened a PR to fix that in repo file: Warn pre-2.2.0~beta2 beta-testers of opam 2.2 about GNU patch on macOS by kit-ty-kate · Pull Request #26061 · ocaml/opam-repository · GitHub
Returning to this thread, because I’m still mystified about what I should be doing in response to this WARNING message from opam.
jhw@peace:~ λ opam --version
2.1.6
jhw@peace:~ λ brew info gpatch
==> gpatch: stable 2.7.6 (bottled)
Apply a diff file to an original
https://savannah.gnu.org/projects/patch/
Installed
/opt/homebrew/Cellar/gpatch/2.7.6 (10 files, 351.4KB) *
Poured from bottle using the formulae.brew.sh API on 2024-01-10 at 16:12:22
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/g/gpatch.rb
License: GPL-3.0
==> Analytics
install: 1,204 (30 days), 3,152 (90 days), 13,718 (365 days)
install-on-request: 277 (30 days), 779 (90 days), 3,402 (365 days)
build-error: 0 (30 days)
jhw@peace:~ λ brew install gpatch 1 ↵
==> Downloading https://formulae.brew.sh/api/formula.jws.json
############################################################################################################################# 100.0%
==> Downloading https://formulae.brew.sh/api/cask.jws.json
############################################################################################################################# 100.0%
Warning: gpatch 2.7.6 is already installed and up-to-date.
To reinstall 2.7.6, run:
brew reinstall gpatch
jhw@peace:~ λ opam update -aR
<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><> 🐫
[conjury.org] no changes from hg+ssh://jhw@prime.conjury.org/hg/project/opam-repository
[bitbucket.org] no changes from git+https://bitbucket.org/jhw/opam-personal
[default] synchronised from https://opam.ocaml.org
[WARNING] Invalid patch utility. Please install GNU patch
Now run 'opam upgrade' to apply any package updates.
jhw@peace:~ λ
I’m running the latest macOS release, with the latest Xcode tools, I have updated HomeBrew, it says I have GNU patch installed, and yet opam continues to give a WARNING that my patch utility is “Invalid” and requests I install GNU patch, which I have installed using the most popular package manager for GNU tools on macOS today, which is the same one I used to install opam itself.
I want to respond properly to this warning, but I still don’t understand what risks I’m assuming by not doing anything and waiting for the next release of something I’m using (not even clear what) that I might hope will fix whatever problem I’m being warned about.
That’s why I came here to ask: what is the hazard this WARNING is meant to bring to my attention, and what am I risking by not doing whatever it is that I should have done instead of what I did, which was to follow the request and to install GNU patch, and why didn’t installing GNU patch the way I did satisfy opam?
I get that Linux and Windows are getting a lot of attention from OCaml developers compared to the poor bastards like me still using macOS, but I’m begging you: please give us a little clarity about what this means?
Thanks!
The tone is unnecessary, thank you.
I think what’s still going wrong is that the patch binary found in PATH is not the one installed via homebrew - what’s command -v patch or which patch giving (still /usr/bin/patch)?
@kit-ty-kate has opened gpatch: add gpatch binary to avoid conflict with macOS patch by kit-ty-kate · Pull Request #174687 · Homebrew/homebrew-core · GitHub which, once merged, should allow the use of gpatch as the first command to try on macOS, which I think should alleviate this.
Agree that we should discuss this cordially without heightening the emotional level, but I think there is a very real problem here that is not specific to the details of this patch/gpatch thing — the fact that large changes to opam’s behavior are being released without anyone even trying it out on a macOS environment (or so it would appear).
I understand that homebrew’s practices are often (a lot) less than ideal, and often defy the expectations of developers and software distributors on unix platforms. Unfortunately, that is where we are today, and I think we need to come up with a solution to the inscrutable breakages that occur in the OCaml/opam ecosystem on the macOS platform — keeping in mind that these breakages are not typically opam’s fault, but ultimately end up being opam’s responsibility.
Regaining some perspective might help here - a spurious warning does not constitute a “large change” (the removal of the files/ directories in opam-repository itself is another matter).
Appearances are apparently deceptive, given that two of the three of opam’s regular maintainers use macOS! Having followed Homebrew’s recommmendations, probably a long time ago, my shell puts /opt/homebrew/bin first in PATH, and I’m guessing that @jhw has a non-standard or, at least, against-the-recommendation configuration of Homebrew, hence the request for the output of command -v patch.
Given that the hypervisors on most of my various machines from down the years are riddled with VMs for exotic and unusual OSes I’ve temporarily installed to debug build issues in both OCaml and opam, these discussions be much better if they started from “I have an environment configured differently from the ones the maintainers considered” than either “they don’t care” or “they didn’t even test it”!
Naturally, I am referring to both the opam software and the opam package repository when I speak of large changes that seem to have passed through without adequate testing on macOS. I can’t speak to the homebrew configuration of the other colleagues in this thread, but I will just say that the combination of breakage and inscrutable messages occurred on plenty of standard configurations of homebrew and opam, and that what happened here very plainly could not have happened if there were adequate testing. I understand that there is nothing that can be done to deal with people’s nonstandard configurations, but I experienced breakage on two different machines with fully standard and up to date configurations — and I had to come here to interpret the error. This speaks for itself.
Anecdata, but this macos was able to update to opam without issue. Without this topic I wouldn’t be aware of the “large change”.
Why is the assumption that no testing was done on Macs at all?
We all know how no two machines are equal, and some edge cases can be hard to anticipate. Given the prices of these machines, I’m not expecting open source projects to keep a battery of them to test their releases.
How many of us, mac users, are actively testing the various release candidates of the ecosystem tools to help spot these bugs? (I know I don’t and I’m grateful for the ones who are.)
I apologize for my tone. I’m highlighting this excerpt from what @jonsterling writes, because it does illustrate the source of my consternation, which I regret expressing. I’ll try not to do that again.
In the interests of providing more clarity, I should make clear that I don’t want HomeBrew to replace my system patch with the GNU clone, and I’m actually happy that HomeBrew appears to have responded to OPAM having a dependency on it by instead installing it in /opt/homebrew/Cellar/gpatch/2.7.6/bin/patch with symbolic links to that from /opt/homebrew/opt/gpatch/bin/patch and /opt/homebrew/var/homebrew/linked/gpatch/bin/patch.
It did this without my noticing. I would never have installed gpatch expressly, and I neither want nor need to have /opt/homebrew/opt in my $PATH.
My shell uses the recommended logic for adding HomeBrew to the $PATH variable, i.e. eval "$(/opt/homebrew/bin/brew shellenv)", which produces this:
jhw@peace:~ λ brew shellenv
export HOMEBREW_PREFIX="/opt/homebrew";
export HOMEBREW_CELLAR="/opt/homebrew/Cellar";
export HOMEBREW_REPOSITORY="/opt/homebrew";
export PATH="/opt/homebrew/bin:/opt/homebrew/sbin${PATH+:$PATH}";
export MANPATH="/opt/homebrew/share/man${MANPATH+:$MANPATH}:";
export INFOPATH="/opt/homebrew/share/info:${INFOPATH:-}";
Note: that places /opt/homebrew/bin at the end of my PATH so that its contents do not override my system tools (because those are added to the front by zsh itself from /etc/paths), which is absolutely what I want and I’d be unhappy if it didn’t do this.
If HomeBrew had installed gpatch in /opt/homebrew/bin— which it did not— then it would have been found there like the other things installed by HomeBrew, e.g. opam:
jhw@peace:~ λ command -v gpatch
jhw@peace:~ λ command -v opam
/opt/homebrew/bin/opam
For what it’s worth, I would not characterize this new behavior of opam as a “large change” but rather as a small change that nevertheless illustrates the issue I was hoping could be resolved in this discussion thread: that opam is issuing a WARNING that, if I understand correctly, indicates that I’m risking some unspecified harm if I don’t agree to replace the system patch tool in my $PATH with the GNU clone, which seems like a pretty intrusive demand for opam to be making, and I want to understand the reasoning for it.
@kit-ty-kate has opened gpatch: add gpatch binary to avoid conflict with macOS patch by kit-ty-kate · Pull Request #174687 · Homebrew/homebrew-core · GitHub which, once merged, should allow the use of
gpatchas the first command to try on macOS, which I think should alleviate this.
As noted above, HomeBrew does not install gpatch into the user’s path when it’s a dependency. It goes into the “Cellar” so that reverse dependencies, e.g. opam, have it available, but it’s only linked into the user’s PATH when the user expressly asks for it. Which I could optionally do with brew install -f gpatch, but I’d like to know the reasoning behind why I have to do it.
Accordingly, I’m not confident the patch above will alleviate any problems, and I wonder if it isn’t too much to ask for somebody on the opam development team to test it.
Thanks for the technical explanation of how homebrew operates. I personally had no idea about that cellar feature to allow packages to install dependencies just for them and not for the end user. This is something I wish opam had as well 
I believe that this tone undermines your message. Also can I remind you of this bit of our code of conduct:
Be respectful. Not all of us will agree all the time, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack.
Thanks.
I think there might be some miscommunication or misunderstanding of @jhw’s tone here — I did not interpret the quoted content
as being disrespectful or exhibiting poor manners (there was a comment earlier on that was, but he has apologised for this). My interpretation of this was a respectful question as to whether any of the very busy opam maintainers would have a spare moment to test out whether this patch does in fact solve the problem that it was claimed to solve in this thread, in light of @jhw’s explanation of homebrew’s behaviour. (There may be multiple ways to interpret “isn’t too much to ask”, but I naturally use the one that assumes good faith.)
In a multinational forum like this (which may have some language barriers) we do indeed have to be especially careful, on all sides, to speak with each other in a way that is least likely to cause misunderstanding or bad feelings — and there is a corresponding duty to assume the best intentions and not frivolously call out fully respectful language as being a potential violation of the code of conduct unless there is some real evidence of bad-faith engagement.
Thanks for your message. I agree that reading subtext in a foreign language is difficult, and you’re right that I interpreted this as a slight escalation in tone; my apologies if that wasn’t the case.
If that aside is now over, let’s get back to the technical discussion.
At present, it doesn’t install gpatch at all, what does command -v patch (and which -a patch give?
This GitHub Actions run is working as we saw locally - brew install opam results in patch being shadowed, which is consistent with what was expected when the dependency on gpatch was added 4 years ago in opam: depend on gpatch by avsm · Pull Request #64301 · Homebrew/homebrew-core · GitHub
As one could infer from my previous messages, it gives the patch installed by the Xcode tools, which is what I am insisting is the correct behavior, and I continue to be mystified at the implication that I should not expect this after installing opam with HomeBrew.
Although, it’s worth noting that I had not noticed the presence of /opt/homebrew/bin/patch as a link to the GNU clone installed by HomeBrew.
jhw@peace:~ λ command -v patch
/usr/bin/patch
jhw@peace:~ λ which -a patch
/usr/bin/patch
/opt/homebrew/bin/patch
jhw@peace:~ λ /usr/bin/patch --version
patch 2.0-12u11-Apple
jhw@peace:~ λ /opt/homebrew/bin/patch --version
GNU patch 2.7.6
Copyright (C) 2003, 2009-2012 Free Software Foundation, Inc.
Copyright (C) 1988 Larry Wall
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Larry Wall and Paul Eggert
jhw@peace:~ λ
One possible reason for the disconnect here may be that I’m using the shell that macOS users are assigned by default in relatively recent OS releases, i.e. zsh, which maybe has somewhat (importantly) different behavior with regard to the composition of its executable search path? That’s why /usr/bin appears first in my search path before /opt/homebrew/bin (as I insist it should), and why I am growing increasingly and tiringly repetitive in asking why opam should require me to replace the version of patch that my shell finds with the GNU clone rather than one installed by the system.
p.s. I’m growing a bit weary of having my tone policed while having my seemingly quite reasonable question go unanswered. If the intention of the opam developers is that I should not have to replace the patch utility installed by my system with the GNU clone, it should not require me to ask four times in a row to prompt an affirmative answer.
p.p.s. Is it possible that my concerns about having to replace the system patch utility in my search path with the GNU clone are being silently dismissed as unreasonable for some reason?
Thank you. This answers my confusion in your previous answers and allows me hopefully to answer all of your questions. You see, you wrote:
which to me at least appeared to be suggesting that when the gpatch formula was being installed as a dependency of opam, it was somehow not being linked from /opt/homebrew/bin (that there was some difference between brew install gpatch and the opam formula depending on the gpatch formula) - but you then also wrote:
which was all the more confusing, because the gpatch package at the moment never installs a binary gpatch - that’s the point of the PR I had originally linked. So, via a roundabout route, I believe we confirm that what I originally wrote:
would indeed result in gpatch being in your PATH (via /opt/homebrew/bin at the end), even if it’s only been installed as a dependency of brew’s opam formula (unless there’s still something I’m missing from both reading Homebrew’s manual and experimenting on two different macOS systems?)
Now, to your original questions:
No, not being silently ignored and I’m sorry if that was the impression - I was simply trying to process the other information you’d included (which didn’t make sense to me, and also didn’t initially answer the question I actually asked). I’m somewhat old fashioned and prefer not to answer things before I actually understand what’s going on.
opam has always required a set of tools to be resolvable in PATH (patch being one of them); this has simply tightened that requirement. I think it’s perfectly reasonable for opam to expect to be executed in an environment conforming to that - of course, you can choose to configure your system such that this only happens when you call opam itself, rather than having to permanently alter your terminal. That’s in just the same way as we recommend the various shell hooks and eval $(opam env), but you’re still welcome to use opam exec.
@mseri answered this a few hours after you posted it three weeks ago:
Now, I certainly think that we should clarify that explanation with the warning emitted by opam update to include the explanation. The issue is that Apple patch will fail to delete files and opam update will fail entering a state where you need to make a manual adjustment. It has meant quite a few instances on opam-repository of reverting the deletions of files from the repository.
This is what I’ve been trying to drill into, and I think we have it. We were assuming that the default configuration for Homebrew on macOS would put homebrew’s /opt/homebrew/bin at the beginning of PATH and thus the advice to install the gpatch formula would shadow Apple’s patch and fix the problem. It turns out that thinking of ours was a bug. Assuming homebrew accepts the suggestion that the gpatch formula should also install a gpatch binary/link, then this problem disappears, since Apple don’t install a gpatch binary and so gpatch should resolve whatever you’ve done to your PATH, as it does on FreeBSD (as @hannes noted).
Hi. Yes, I think we have reached alignment on what is the technical problem I’m observing. I am, however, not yet swayed that we have a consensus about the higher-level problems I came here to discuss.
It’s worth noting that FreeBSD, unlike macOS, ships with its own built-in application package management software: pkg(7). The convention adopted there is for GNU clones of the FreeBSD system compiler toolchain are installed with names prefixed by g to signify the GNU version when both are visible in the PATH executable search. While HomeBrew has adopted that convention with its make package, which installed GNU Make renamed as gmake, I think it’s illustrative to my higher level concern to ask, “What if the HomeBrew people take a different view?”
I can’t help but think about how this played out. When opam developers decided years ago to adopt a dependency on the patch in the system toolchain, it seems that they assumed that patch would always be the GNU patch (or some functional equivalent for all their purposes). Later, when the disparity between GNU patch and the patch delivered in other system toolchains had problematic consequences under opam, the developers could have chosen to adapt opam so that it could use the native patch tool provided by the system toolchain for the platform, but they didn’t do that. Instead, they decided to force a narrow dependency on the GNU version of the patch tool and require that it be installed alongside the system toolchain. Seemingly without testing that the dependency is met in the common default cases.
I think that was a choice that illustrates a prioritization that I feel could be opened to question.
Moreover, when I came into this thread to discuss the specific technical issues I was seeing on macOS with the HomeBrew application package manager, the main disconnect that we had was that I was questioning the decision for patch to be replaced with the GNU clone in the executable search PATH in my interactive shell. As the thread above shows, it turned out that the way HomeBrew, in its default mode on recent macOS systems, typically makes sure the system toolchain is not shadowed by its own packages was never even considered by the opam developers when they made its HomeBrew packaging logic.
Which is another choice that illustrates a prioritization that I feel could be opened to question.
Meaning: I want to ask, how did it seem normal and natural that a user should just accept that installing opam on their system would entail shadowing in their interactive executable search the patch tool in the system compiler toolchain with something that was never qualified by the system compiler toolchain vendor? And, how did it seem sensible to ship a release of opam that issues a WARNING when that resolves to the system compiler toolchain, calling it an “invalid” patch tool?
I feel like these higher level questions are reasonable, and I hope that my tone in presenting them here is seen to be as collegial and cordial as I am trying to make them.
Isn’t the problem here just with homebrew here ?
I thought it was somehow their policy not to override system binaries by default. You’d always need do it explicitely yourself with brew link.
So I guess the gpatch package could just install gpatch, link it by default since it doesn’t overwide anything and link patch only on brew link (or via another package).
If the homebrew opam package is made dependent on the gpatch package then I think everyone’s happy no ?
Actually, this part I’m still interested to drill into. I’m not at my desk this week, which is where my Mac Mini is - I set-up that machine last year and didn’t get this setup, and while a GitHub Actions runner is not exactly a default configuration, their configuration doesn’t behave this way either. At the moment it remains “unclear” to me exactly what the default is, even if it is clearly an error on our side to have assumed that /opt/homebrew/bin would in general be first PATH.
I agree (and indeed have already agreed) that the message of the warning could be improved. There is a small chance, given the amount of effort that was put into trying to move this issue forwards in 2.1.6 (and during the development cycle for 2.2.0), that we didn’t spend enough minutes thinking about that string! 
In terms of “normal”, to be honest where macOS is concerned it kind of is - make is a similar example, where macOS ships a truly ancient version of make and unconditionally installs it in a privileged position. The key thing for the specific suggestion is that while there are myriad examples of where BSD’s (and Apple’s) patch doesn’t work, I’m not aware of any instances of a patch file that is processed “correctly” by BSD(/Apple) patch but processed incorrectly by GNU patch.
That said, a lot of this was discussed, and I’ll give the sea of links because I feel it attempts to address your prioritsation question:
All with quite a lot of discussion and open future work to try to improve things further. In those issue and PR discussions, there are indeed suggestions that we could both avoid need patch entirely and also whether the precise command could be configurable.
The homebrew opam package was made dependent on the gpatch package four years ago in opam: depend on gpatch by avsm · Pull Request #64301 · Homebrew/homebrew-core · GitHub! However, it installs the binary only as patch.
And the followup patch from @kit-ty-kate has already been submitted and is under review at Homebrew: gpatch: add gpatch binary to avoid conflict with macOS patch by kit-ty-kate · Pull Request #174687 · Homebrew/homebrew-core · GitHub. That’ll bring a gpatch into scope in the PATH and solve the problem in this thread.
More testing reports on that PR would help the Homebrew maintainers determine whether to merge this.