I honestly have no idea what to do with this information. What I do know is that this kind of thing should not be occurring on an OCaml installation that has been working fine — surely there should be some user-facing and user-fixable issue reported before getting to this point. I would ordinarily “solve” this problem by nuking my .opam folder, but I would like to raise this as a usability issue in here and see what can be done — and learn what I ought to have done to prevent this from happening, if anything.
You’re likely using a macOS system with opam < 2.1.6 and patch being not a GNU patch. (This results in files aren’t deleted, but only emptied, and kept around).
To recover, rm -rf ~/.opam/repo/default{,.tar.gz} && opam update default should bring your local opam-repository into a good shape.
I tried this and I still the see the problem. It’s indeed on macOS. Not a great experience if the opam policy change leads to this and will likely affect a lot of users.
Thanks for your report. I have some further questions:
did ~/.opam/repo/default exist?
does ~/.opam/repo/default.tar.gz exist?
In case you still have an errorneous state, could you remove the ~/.opam/repo/default.tar.gz (if it exists), do a opam update and then whatever you intended to do, please?
Thanks for your comment. I actually use FreeBSD since a long time. I do not think it is really relevant here.
If you like to go deeper into detail, let’s take a look at “opam”, and it’s code using patch:
In version up to 2.1.5 (included), the following code lines may be useful to read: opam/src/core/opamSystem.ml at 2.1.5 · ocaml/opam · GitHub
So, for FreeBSD and OpenBSD nothing changed (apart from the extra check whether “gpatch --version” outputs something with GNU patch) – the binary named “gpatch” gets called. On macOS, the prefered binary is still “patch”, but “gpatch” is a fallback, and this fallback is active if “patch --version” doesn’t say “GNU patch”. Also, there’s a warning being issued.
TL;DR: yes, on BSD systems the default patch is not GNU patch. opam deals with this since a long time. The tricky part is macOS, where as far as I know different package managers (homebrew / macports) install gnu patch as patch or gpatch.
Okay, so that’s opening more questions than it answers.
You’re likely using a macOS system with opam < 2.1.6 and patch being not a GNU patch.
I encountered this today using opam == 2.1.6 and patch == patch 2.0-12u11-Apple (which is /usr/bin/patch installed by Xcode tools). And I was seeing the warning that patch is not GNU at the same time I was seeing the gen_ocaml_config faillure.
Now I come to learn that the warning is “not really relevant here” and that leaves me confused. I wasn’t using opam < 2.1.6, so I assumed the problem was that my patch was the one installed by Xcode and not GNU patch, which made me think, “Hey, did the opam people decide that I have to install GNU patch over my Xcode tools?” Because that’s what the warning seems to be telling me. Am I wrong to read it that way?
If there is a warning that I’m using the Xcode tools and not the GNU tools, then what should I do about it? If it’s not relevant to the gen_ocaml_config problem that brought me here, then what is it relevance? What is the problem that would be solved by me installing GNU patch over the top of my Xcode tools?
Please excuse me, I was refering to “FreeBSD”. It was poorly worded from my side.
I hoped that the code would be clear: in opam 2.1.6 if you have GNU patch installed as “gpatch”, this will be used. Would you mind to try with your favourite packaging system (or from source) to install a gpatch binary and take a look whether the warning from opam persists?
Let’s assume for the moment that it would not persist, which I’m inferring is what you would expect.
The question I now have is why opam is presenting me with a warning that it must “fall back” to the default system patch (installed on macOS as patch 2.0-12u11-Apple by Xcode tools and as patch 2.0-12u11 FreeBSD on FreeBSD 13.3).
Why is this even a warning? What is the hazard that I’m risking by not installing the GNU patch over the top of the one installed by my system tools? And moreover, why am I being warned that my default system tools are not what opam is expecting to find unless opam is not expecting to run on systems that are not GNU/Linux systems?
I think you have a valid concern. The big issue with Apple patch is that it is unable to remove files, and instead simply zeroes them. Luckily the (annoying) solution is to nuke the opam folder and reinstall, so in those rare instances one loses a “few” minutes but it is not the end of the world (at least if we find a way to communicate it properly).
But this will not affect just this change of policy (which is a one-time thing): the same reason did break opam switches on macos in the past every now and then when files were removed, leading to the complete stop in removing content from the repository, and it needs to be dealt with in the future when, to slim and cleanup the repository, we will move stale and uninstallable opam files to a separate opam-repository-archive (see Requests for comments: how does opam-repository scale? · Issue #23789 · ocaml/opam-repository · GitHub, there should also be a thread about it on this forum).
This is also why the warning was introduced and we made sure that the main distributions for macos (which is the only affected system in practice), namely homebrew and macports, are bringing gpatch as a dependency.
True, this is not covering all the macos users for now, since who does manual installs like me gets just the warning, and it was not possible to solve differently in opam 2.2 unfortunately (afaik, there were attempts but all failed for different reasons). A true solution, not caring about patch, is already in the work for after 2.2 is released but I don’t know what could be the timeline there.
As per the “why is it just a warning”, I am guessing here that it is because things work with the dafult apple patch in most cases and only in the rare occasion files are deleted you will notice a problem.
What is the solution for opam on macOS? The initial suggestion of rm -rf ~/.opam/repo/default && opam update default did not fix it. Updating to the latest opam also did not do it. I am a bit surprised that the “solution is to nuke the opam folder and reinstall” is offered with a straight face for something as central as opam.
Ideally the solution would be to have gpatch installed. If you install opam with macports it is enough to run a selfupdate and an upgrade, homebrew already depended on gpatch since a while. Manually I don’t know what is the best way to install gpatch on mac.
EDIT: I tried deleting gpatch, loading an older version of the repo and opam updating it to make it fail. Somehow to me the suggested solution worked for me (and I could not find the default.tar.gz file in the folder before or after the update)
I am thinking that opam may make some announcement mechanism: when user run opam update, it can prompt a warning and fix like this for users at specific platforms.
For people using opam 2.1.6 there is already a big warning when GNU patch is not detected, and for people using opam 2.1.5 on macOS there should also be a warning:
Were the warnings not displayed? If so, there is a bug somewhere
It now shows I am using opam 2.2.0~alpha3 but I think I was using the previous alpha version at 2.2.0.
I happened to remove the brew-installed ocaml and opam on the same day. I am using the manually installed opam and the opam-installed ocaml, so I have never used the brew-installed ones. I may see remove gpatch in a glance of the brew removing logs. The mac is in some transitional state for gpatch. So it may be my fault rather than opam.
I don’t have gpatch installed when running the fixing command.
(I’m on a MacBook Pro M3 Max).