Windows compiler support in opam 2.2.0~beta2

Don’t worry - it’s enough to be trying to reproduce it, thanks!

1 Like

For reference, I am trying to compile forester.
I have the repo checked out at commit 897258915637265914a117da53bf2af38a58bbad since later commits added a package which relies on pkg-config. Working around the issue regarding the ssl dependency, I managed to successfully install all necessary packages and am now encountering the following error running dune build:

❯ dune build
File "test/dune", line 2, characters 8-13:
2 |  (names parse)
            ^^^^^
<3>WSL (65808) ERROR: CreateProcessEntryCommon:505: execvpe /bin/bash failed 2
<3>WSL (65808) ERROR: CreateProcessEntryCommon:508: Create process not expected to return
** Fatal error: Error during linking

File "caml_startup", line 1:
Error: Error during linking (exit code 2)
File "bin/forester/dune", line 3, characters 7-11:
3 |  (name main)
           ^^^^
<3>WSL (65814) ERROR: CreateProcessEntryCommon:505: execvpe /bin/bash failed 2
<3>WSL (65814) ERROR: CreateProcessEntryCommon:508: Create process not expected to return
** Fatal error: Error during linking

File "caml_startup", line 1:
Error: Error during linking (exit code 2)

A little strange that WSL appears in the error message…

A little strange that WSL appears in the error message…

If you look at your where.exe bash output you will see the first entry is C:\Windows\System32\bash.exe … that is WSL2 bash.exe. All dune build is doing is executing the shell that is in the PATH, which is the WSL2 bash.exe.

Could you try instead:

opam exec -- dune build

That should adjust your PATH to make shells available that are suitable for opam 2.2.

1 Like

It gives the same error, but I will investigate the issue with the path. Thanks!

A couple of things, having tried locally: I can reproduce the original error simply by installing mingw-w64-ucrt-x86_64-gcc and putting MSYS2’s ucrt64 bin directory in PATH, but only when using the not-yet-upgraded default remote (i.e. running opam init vs opam init git+https://github.com/dra27/...) … did you just run opam init on its own the first time? Using the newly upgraded packages from my windows-initial this works (which is what I’d expect).

The next part is a subtle bug - I think the problem here is a long linking command and flexlink is trying to use bash to avoid overly long command lines. The fix for you for now should be to put C:\Cygwin64\bin before C:\Windows\System32 in your PATH, but I’ll need to scratch my head a bit as to what the full fix ought to be…

Yes, this probably happened cause I was reinstalling a bunch of times. Good to have this confirmed.

Rearranging the path worked and I have successfully compiled, although for anyone who comes across this: Cygwin added itself to the path in my user variables, so I needed to move that variable over to the system variables.

Thanks!

1 Like

Just to clarify (also for anyone coming across the thread) - the internal Cygwin (or eventually internal MSYS2) installation is the recommended approach, not having to install your Cygwin! The original issue you saw (by running opam init without the repository) will be unhittable once the opam-repository PR has been merged (i.e. that issue is a consequence of being a beta tester).

I have a fix in mind for flexlink which should fix the ** Fatal error: Error during linking, which is being tracked in ocaml/flexdll#138.

Many thanks again both for trying it all out and reporting the issues!

2 Likes

Hello,

I finally had some time to try opam on Windows. I’m not sure if issues should be reported here or in a specific Github; I can copy them there if this is not the best place.

On a Windows 10, I installed Git for Windows, WinGet, and from the Git Bash terminal, ran the command:

opam init git+https://github.com/dra27/opam-repository.git#windows-initial

A very minor detail: the first prompt showed this:

Which Git should opam use?
> 1. Use default Cygwin Git
  2. Use found git in C:\Users\user\AppData\Local\Programs\Git\mingw64\bin
  3. Use found git in C:\Users\user\AppData\Local\Programs\Git\mingw64\bin
  4. Use found git in C:\Users\user\AppData\Local\Programs\Git\cmd
  5. Enter the location of installed Git
  6. Abort initialisation to install recommended Git.

I wonder why it reported twice the same path (C:\Users\user\AppData\Local\Programs\Git\mingw64\bin, 2 and 3).

I chose 1, and then 1 again for “How should opam handle Cygwin?” (> 1. Automatically create an internal Cygwin installation that will be managed by opam (recommended)). But then I got:

Fatal error:
OpamDownload.Download_fail(_, "curl: code  while downloading https://cygwin.com/sha512.sum")

Note the lack of the error code (there are only 2 spaces) in the error message.

which curl says /mingw64/bin/curl, so I guess the issue is that the curl executable is the one that Git Bash uses, which might not be the right one… running opam init from a Command Prompt seems to work fine.

So, just to add another failure mode, and possibly a suggestion for the documentation: do not use Git Bash to initialize opam… I guess this is obvious once you realize that Git Bash uses MSYS2 (at least that’s what I think), and that MSYS2 is incompatible with Cygwin. Or maybe this is improbable enough that does not warrant being mentioned.

1 Like

By the way, sorry about arriving late, but concerning the pkg-config issue and packages relying on it (in particular, I depend on conf-zlib), I wonder if there is an estimation of how long it might take? Otherwise there’s not much I can currently test on Windows… If there is something I can do with the conf- packages myself, I can try to do it, to speed things up.

I didn’t check the details, but in our project we use pkgconfiglite. It appears dead but works well for many years.
Just simple command should make pkg-config work on Windows:

choco install -y pkgconfiglite --download-checksum 2038c49d23b5ca19e2218ca89f06df18fe6d870b4c6b54c0498548ef88771f6f --download-checksum-type sha256

I’m glad it’s working in the context you’re using it, but in general installing (semi-random) very old alternate Unix packages is at best just going to fudge things. In particular, given that libraries are coming from cross-compiled environments (mingw-w64 is a cross-compiled environment from Cygwin), I’m not sure how a drop-in alternative for pkg-config is going to deal with the missing command line flag in the opam packages:

e.g. Cygwin’s pkg-config, which is a symlink gives:

$ pkg-config -libs libzstd
Package libzstd was not found in the pkg-config search path.
Perhaps you should add the directory containing `libzstd.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libzstd', required by 'virtual:world', not found

(the same can be seen using pkgconf, which is not a symlink)

verses the correct (for 64-bit mingw-w64):

DRA@Tau ~
$ pkg-config --personality=x86_64-w64-mingw32 -libs libzstd
-lzstd

Thanks for the report and testing, @amaro!

This menu is getting one or possibly two overhauls in beta3 - in particular, if the git.exe found in PATH is Git for Windows, this menu is no longer displayed at all (which is what should happen here). This appears to be a consequence of running from within Git Bash which, for me, gives a strange PATH:

DRA@Tau MINGW64 ~
$ echo $PATH
/c/Users/DRA/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/DRA/bin:<my standard PATH with C:\Program Files\Git\cmd translated to /cmd>:/usr/bin/vendor_perl:/usr/bin/core_perl

and thus a similar three possible equivalent results for git:

DRA@Tau MINGW64 ~
$ which -a git
/mingw64/bin/git
/mingw64/bin/git
/cmd/git

so it makes sense that it’s included it twice, but I’ll get that fixed regardless.

Ouch! Cygwin and MSYS2 are indeed completely incompatible if they start trying to load each other’s libraries and I believe that things can get extremely hairy if you’re mixing the shells, but opam as a native Windows executable should be able to call both. I have a hunch as to what’s going on here, but I’ll need to reproduce it locally first.

Oh no - the fact you did it means that opam should at the very worst gracefully complain! code while is not a graceful complaint :slightly_smiling_face:

Sooner rather than later - the main thing is having ocaml/opam-repository#25861 merged, as that finalises the layout for updating the other conf- packages.

#25861 contains a commit which updates conf-zstd (and see also the already-merged conf-mingw-w64-zstd-i686 and conf-mingw-w64-zstd-x86_64).

conf-zlib will want updating in a very similar way. If you’re trying to do this before #25861 is merged, then I’d suggest creating a new branch from my windows-initial branch and adding commits on top of that and then testing with your fork. Once #25861 is merged, upgrades to individual conf- packages can be submitted as PRs (and will be most warmly welcomed!).

Again, just in case the thread is encountered in future: while running opam init from git-bash is not really recommended[1], its curl should work with the next opam beta, once Harden curl output parsing by dra27 · Pull Request #5984 · ocaml/opam · GitHub is merged.


  1. git-bash is intended for interacting with Git - it’s why, for example, it doesn’t include pacman ↩︎