OCaml 5 seems to sort of support Windows, vaguely. As far as I can see it uses an internal cygwin environment so it’s still keeping Windows at arms length. Presumably that is because so many packages assume Unix.
I’m trying to build one of those packages. It doesn’t use Dune, and its Makefile only works in a POSIX-like environment (it uses rm -f, /dev/null, etc.).
When I install it via opam install . it works fine, presumably because it runs it in some kind of cygwin shell. However if I just make install then I run into the Makefile compatibility issues.
I want to debug something in the build process, so my question is how can I run make install in the same way that OPAM does. I’m imagining something like opam exec cygwin or similar that would put my in some kind of cygwin shell where /dev/null exists and rm -f works.
The OCaml compiler itself requires a posix-ish environment to be built (configure script, makefile) but note that by default, all the resulting binaries are windows native binaries.
I haven’t tried, but i think opam exec -- bash should work. If not, the builtin Cygwin installation is located in C:\Users\<user>\AppData\Local\opam\.cygwin (aka. %LocalAppData%\opam\.cygwin) and you should be able to find the Cygwin terminal emulator somewhere in there
Yeah I did try that but it actually starts Bash in WSL! I think Microsoft has aliased bash to do that.
So I tried opam exec -- “C:/Users/Tim/AppData/Local/opam/.cygwin/root/bin/bash.exe” and that does start a cygwin Bash shell, but I don’t have rm or ls available. Seems like it doesn’t set PATH correctly because they are in that bin directory. Hmm.
As @kit-ty-kate notes, all the binaries produced are indeed native Windows binaries. The rationale behind Windows opam is that one is trying to produce native Windows software, and therefore using native Windows tools, so the internal Cygwin environment is there for one’s dependencies but not intended for development. Debugging a broken dependency unfortunately sits in a grey area, therefore!
You can indeed run bash -l to use the internal Cygwin installation. It’s also possible to install Cygwin as normal (e.g. to its default C:\cygwin64) and to instruct opam to use that installation, instead of managing an internal one. There’s nothing wrong with using or editing the internal Cygwin installation, the only caveat is that opam doesn’t prompt when making changes to an internal installation, but it will ask before making changes to your own external Cygwin or MSYS2 installation.
so the internal Cygwin environment is there for one’s dependencies but not intended for development
Ahhh interesting. I think that’s where the thing I’m debugging is going wrong then. It’s a compiler that embeds an absolute path to its standard library at build time. Because it’s a dependency, it’s running in cygwin so it finds this path: /cygdrive/c/Users/Tim/AppData/Local/opam/default/share/lem/library, but then in another project when it runs lem I guess it isn’t running in cygwin so that path doesn’t exist - it doesn’t look in C:\Users\Tim\AppData\Local\opam\default\share\lem\library. That sucks. I can’t think of an immediately obvious non-hacky solution either.
I don’t quite understand why that happens though because the other project is also a dependency…
I’m trying to use the setup-ocaml Github Action to test CompCert under Windows, and I’m running in the problem discussed here, without a workable solution so far.
CompCert has a Bourne shell script tools/runner.sh that handles all the CI steps. The problem is how to call it from a GHA:
opam exec -- C:/.opam/.cygwin/root/bin/bash.exe tools/runner.sh <arguments> fails because PATH is not initialized to include /bin, /usr/bin, etc.
opam exec -- C:/.opam/.cygwin/root/bin/bash.exe -l tools/runner.sh <arguments> fails because bash -l changes the current directory to $HOME (i.e. C:/Users/runneradmin for GHA) instead of the directory where the CompCert sources were checked out (somewhere in D: for GHA).
This is really testing my patience. How hard can it be to run a shell script?
Ideally, opam exec would 1- call bash automatically (for compatibility with Linux and macOS builds), 2- with a properly set up PATH, and 3- without changing the current directory.
This behaviour is suppressed if you set CHERE_INVOKING prior to the call (it’s part of Cygwin’s own functionality for its explorer shell integration in the chere package; MSYS2 makes the mechanism even more explicit):
That said, the directory for the checkout is in the GITHUB_WORKSPACEenvironment variable (i.e. cd "$GITHUB_WORKSPACE" should also put the script straight into the correct place, even without CHERE_INVOKING shenanigans).
I realise this isn’t helpful here, but that’s not the philosophy behind the design of opam exec - it’s designed to integrate with the actual shell being used (which is almost certainly PowerShell here). The ability to execute shell scripts is available in build and install sections for packages to increase “portability”, but that’s not exposed to the native Windows user via opam exec.
The closer behaviour here would be to install Cygwin directly in the action, as OCaml’s build-msvc.yml workflow does, and override the shell (see Blaming ocaml/.github/workflows/build-msvc.yml at trunk · ocaml/ocaml · GitHub). i.e. opam very intentionally sees a complete difference between being executed from Cygwin’s bash versus from a native Windows shell. OCaml’s workflow takes complete control of opam, but I think you can also tell the setup-ocaml action to use a separately installed Cygwin installation.
Thank you, David, for the CHERE_INVOKING trick. I wouldn’t have guessed
Concerning opam exec, I think what I was trying to say is that Opam for Windows seems to “just do the right thing” when running “build” and “test” actions, even if these actions use Posix shell scripts and commands, but unsophisticated users like me are unable to reproduce these actions because we have no clue how to “do the right thing” in a Windows environment to run the same Posix shell scripts. Is there a way to ask Opam to put me in the very same environment it uses to run “build” and “test” actions?
At any rate, Opam for Windows + a manual installation of Cygwin works well for my interactive testing, but I’m giving up on turning it into a Github CI workflow (too difficult to automate and too costly to run).
