Issue between Opam2.2.1/Cygwin/Windows and C

Ecosystem
,
1

I am trying to install core with opam2.2.1 on Cygwin.

When compiling base.v0.16.3, it fails because x86_64-w64-mingw32-gcc.exe is not found.

An easy fix: create a /usr/local/bin/x86_64-w64-mingw32-gcc.exe.exe symlink.

Now, I have:

# x86_64-w64-mingw32-gcc.exe: fatal error: cannot execute ‘cc1’: spawn: No such file or directory

An easy fix: add /usr/lib/gcc/i686-w64-mingw32/11 in the PATH.

Now #include <malloc.h> does find the file to be included, but fails to include mm_malloc.h

An easy fix: add /usr/lib/gcc/x86_64-w64-mingw32/11/include in the CPATH.

Now, I have some weird errors on assembly level:

https://www.sinerj.org/~loyer/base-1968-61c947.out
The environment is here:
https://www.sinerj.org/~loyer/base-1968-61c947.out

It looks like my gcc installation is broken, but I can easily compile a program which includes malloc.h and uses malloc() without all the workarounds.

2

Your C toolchain does not seem to be correctly installed. Have you installed the x86_64-mingw-w64 toolchain using the Cygwin setup program? Note that the packages prefixed with i686 are 32-bit packages and probably not what you want.

Cheers,
Nicolas

3

My bad, x86_64 instead of i686 fix the last error.

About the first issues, I guess I understand better what happens. The C toolchain is well installed (via the Cygwin setup) and compiles correctly some simple programs.

However, dune try to launch x86_64-w64-mingw32-gcc.exe and Cygwin map the command to x86_64-w64-mingw32-gcc.exe.exe that doesn’t exists.

The x86_64-w64-mingw32-gcc command analyses its argv[0], then a x86_64-w64-mingw32-gcc.exe.exe symlink is broken.

I can have a working x86_64-w64-mingw32-gcc.exe.exe program with a little wrapper:

#include <unistd.h>

int main(int argc, char *argv[])
{
  argv[0] = "/usr/bin/x86_64-w64-mingw32-gcc";
  execvp("/usr/bin/x86_64-w64-mingw32-gcc", argv);
  return 0;
}

But I guess that making dune trying to exec gcc without the .exe suffix would be better.

And now… something else, something which seems to deal with the dune rules:

https://www.sinerj.org/~loyer/base-18688-35ab5d.out

# Error: Rule failed to generate the following targets:
# - hash_types/src/internalhash_stubs.o
# File "src/discover/dune", lines 1-2, characters 0-91:
# 1 | (executables (names discover) (libraries dune-configurator)
# 2 |  (preprocess no_preprocessing))
# Error: Rule failed to generate the following targets:
# - src/discover/.discover.eobjs/native/discover.o
# File "shadow-stdlib/gen/dune", lines 1-2, characters 0-115:
# 1 | (executables (names gen) (libraries str compiler-libs.common)
# 2 |  (link_flags -linkall) (preprocess no_preprocessing))
# Error: Rule failed to generate the following targets:
# - shadow-stdlib/gen/.gen.eobjs/native/mapper.o
4

I don’t know what the issue is, but Dune works perfectly on Windows and the mingw64 toolchain (we use it in production every day). So I would rather double-check the OPAM configuration, whether the right switch is being used, etc.

Cheers,
Nicolas

5

I don’t know what happens, but if I add a /usr/local/bin/x86_64-w64-mingw32-gcc.exe.exe wrapper. It works.

But I have to program the wrapper correctly.

I have made:

Sys.argv.(0) <- "x86_64-w64-mingw32-gcc";

let pid = Unix.create_process
  {|c:\Cygwin\bin\x86_64-w64-mingw32-gcc.exe|}
  Sys.argv
  Unix.stdin Unix.stdout Unix.stderr in

let (_pid,process_status) = Unix.waitpid [] pid in

match process_status with
| Unix.WEXITED(code) -> 
    exit code
| Unix.WSIGNALED(_sig) ->
    print_string "signaled";
| Unix.WSTOPPED(_sig) ->
    print_string "stopped"

(My C wrapper doesn’t work even with Windows path… I guess execvp should be reserved for Unix…).

I have seen in Dune, the addition of .exe for Win32. But

      if Sys.win32 && not (Filename.check_suffix name ".exe") then
        name ^ ".exe"
      else name

SHould not add .exe twice… perhaps there is something nasty in my configuration which makes dune try to find x86_64-w64-mingw32-gcc.exe.exe.

(Note: I had the same issue building base.v0.16.3 from source with dune)

1 Like
6

I still don’t think the problem is with Dune. Can you provide some more information about your development environment? What is the output of opam switch? What is the output of ocamlc -config? How did you install the mingw64 toolchain? etc.

Cheers,
Nicolas

7

MinGW has been installed from the Cygwin installer. Nothing fancy.

Opam switch:

$opam switch
#  switch   compiler
          description
→  5.1.0
          arch-x86_64.1,ocaml-base-compiler.5.1.0,ocaml-options-vanilla.1,system-mingw.1
          ocaml-base-compiler = 5.1.0 | ocaml-system = 5.1.0
   5.2.0    ocaml-system.5.2.0
          ocaml-base-compiler = 5.2.0 | ocaml-system = 5.2.0
   default
          arch-x86_64.1,ocaml-base-compiler.5.2.0,ocaml-options-vanilla.1,system-mingw.1
          ocaml >= 4.05.0

Ocamlc conf

$ocamlc -config
version: 5.1.0
standard_library_default: C:/Users/frede/AppData/Local/opam/5.1.0/lib/ocaml
standard_library: C:/Users/frede/AppData/Local/opam/5.1.0/lib/ocaml
ccomp_type: cc
c_compiler: x86_64-w64-mingw32-gcc
ocamlc_cflags: -O2 -fno-strict-aliasing -fwrapv -mms-bitfields
ocamlc_cppflags:
ocamlopt_cflags: -O2 -fno-strict-aliasing -fwrapv -mms-bitfields
ocamlopt_cppflags:
bytecomp_c_compiler: x86_64-w64-mingw32-gcc -O2 -fno-strict-aliasing -fwrapv -mms-bitfields
native_c_compiler: x86_64-w64-mingw32-gcc -O2 -fno-strict-aliasing -fwrapv -mms-bitfields
bytecomp_c_libraries:  -lws2_32 -lzstd -lversion   -l:libpthread.a -lgcc_eh
native_c_libraries:  -lws2_32 -lzstd -lversion   -l:libpthread.a -lgcc_eh
native_pack_linker: x86_64-w64-mingw32-ld -r -o
native_compiler: true
architecture: amd64
model: default
int_size: 63
word_size: 64
system: mingw64
asm: x86_64-w64-mingw32-gcc -c
asm_cfi_supported: false
with_frame_pointers: false
ext_exe: .exe
ext_obj: .o
ext_asm: .s
ext_lib: .a
ext_dll: .dll
os_type: Win32
default_executable_name: camlprog.exe
systhread_supported: true
host: x86_64-w64-mingw32
target: x86_64-w64-mingw32
flambda: false
safe_string: true
default_safe_string: true
flat_float_array: true
function_sections: false
afl_instrument: false
windows_unicode: true
supports_shared_libraries: true
native_dynlink: true
naked_pointers: false
compression_supported: true
exec_magic_number: Caml1999X033
cmi_magic_number: Caml1999I033
cmo_magic_number: Caml1999O033
cma_magic_number: Caml1999A033
cmx_magic_number: Caml1999Y033
cmxa_magic_number: Caml1999Z033
ast_impl_magic_number: Caml1999M033
ast_intf_magic_number: Caml1999N033
cmxs_magic_number: Caml1999D033
cmt_magic_number: Caml1999T033
linear_magic_number: Caml1999L033
8

Is the mingw-w64-shims package installed in your switch?

9

Yes it is installed. But the symptom is that when compiling with dune, the OCaml compiler is searched with the name ocamlopt.exe (it works), but the C compiler is searched with a double extension: x86_64-w64-mingw32-gcc.exe.exe then it fails (excepted with a wrapper as proposed).

I have an other issue: endian.h is not found on my system. This breaks the Janestreet’s core library. (fixed by installing cygwin-devel and adding /usr/include in the CPATH variable… not sure it is the best thing to do)

Curiously, I had less issues with the opam-2.2.0~alpha2 (perhaps du to the used repository)

10

I’m curious to try to figure out what’s going wrong on your system. Is it possible to get the output of ocamlc -config-var c_compiler - in particular the 5.2 compiler you were using?

The environment file you have reveals various things which are possibly awry - in particular what did you do to get an ocaml-system.5.2.0 switch?

Some bits of hopefully useful info:

  • opam 2.2.0~alpha2 would have almost certainly been using opam-repository-mingw (git+https://github.com/ocaml-opam/opam-repository-mingw), which would have picked up patches for core 0.15. core.0.16 as far as I know definitely doesn’t build on Windows at the moment - it’s needs to be patched.
  • As a reference point, base.v0.16.3 builds with no problem on my “default” 2.2.1 installation (internal Cygwin installation, etc.)
  • I’ve attempted to see what might be fed to Dune to trick it into emitting the double .exe, but so far all I get is Program x86_64-w64-mingw32-gcc.exe not found in the tree or in PATH (which happens if c_compiler is mingw-w64-mingw32-gcc.exe instead of mingw-w64-mingw32-gcc and strikes me as a possible minor piece of hardening).
11

All my switches have a x86_64-w64-mingw32-gcc value as c_compiler config. It is really weird.

NOTE: I can’t reproduce the error when disabling the gcc wrapper on 4.14.2, 5.1.0 and 5.2.0, but have still this issue on default (fixed by the wrapper).

Also, When I have tried to remove base and reinstall it on my 5.2.0, it removed ocaml-system and replaced it by ocaml-base-compiler.

Note: I have managed to build core.v0.16.2 with the endian.h from cygwin-devel (and the CPATH pointing to /usr/include … and a 5.1.0 switch since 5.2.0 changes a Gc API). base.v0.16.2 only needs my gcc wrapper.

12

If the default switch is still doing this, would you be able to post a few more bits about it (I’ve been assuming from what you wrote before that you’re using a Cygwin shell):

which -a opam
which -a dune
opam exec --switch=default -- ocamlc -config
opam list --switch=default
cmd /c "dir $(opam var bin)"

The use of ocaml-system in that switch was already strange to me, but it looked as though your path had somehow become mangled (the switch binary directory appearing twice in it), so that doesn’t unduly surprise me (although the fact it had got this way does).

I’m glad this is working for you, but just to add that ripping headers out from random other distributions and setting environment variables is not the general solution! endian.h is a posix-specific header, and the code in Core needs updating not to use it on Windows (I write this as much for the record in the thread, as anything else)

13

The PATH is strange… multiple opam directories…

$which -a opam
/cygdrive/c/Users/frede/AppData/Local/Programs/opam/bin/opam
$which -a dune
/cygdrive/c/Users/frede/AppData/Local/opam/default/bin/dune
/cygdrive/c/Users/frede/AppData/Local/opam/5.1.0/bin/dune
/cygdrive/c/Users/frede/AppData/Local/opam/5.1.0/bin/dune
/cygdrive/c/Users/frede/AppData/Local/opam/5.1.0/bin/dune

But the c_compiler of the default switch is:

c_compiler: x86_64-w64-mingw32-gcc.exe

While from the other switches:

c_compiler: x86_64-w64-mingw32-gcc

This could explain the difference of behaviour.

I have rebuild Ocaml (opam reinstall ocaml-base-compiler), and now, I have:

c_compiler: x86_64-w64-mingw32-gcc

And opam reinstall base (which uses the C compiler) succeeds to compile without the wrapper.

I guess I had a bad environment while boostraping OCaml with opam init.

Now, if I get rid of my opam directory and rebuild it (opam init), I have the good c_compiler value.

Note, my PATH is really weird.

/cygdrive/c/Users/frede/AppData/Local/opam/default/bin:/usr/x86_64-w64-mingw32/sys-root/mingw/bin:C:.:C:C:/cygdrive/c/Users/frede/AppData/Local/opam/default/bin:.:C:/usr/x86_64-w64-mingw32/sys-root/mingw/bin:C:/cygdrive/c/Users/frede/AppData/Local/opam/default/bin:.:C:/usr/x86_64-w64-mingw32/sys-root/mingw/bin:C:/cygdrive/c/Users/frede/AppData/Local/opam/default/bin:.:C:/usr/x86_64-w64-mingw32/sys-root/mingw/bin:/cygdrive/c/Users/frede/AppData/Local/opam/default/bin:/usr/x86_64-w64-mingw32/sys-root/mingw/bin:/usr/local/bin:/usr/bin:/cygdrive/c/Program Files/Eclipse Adoptium/jdk-17.0.6.10-hotspot/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/WINDOWS/System32/WindowsPowerShell/v1.0:/cygdrive/c/WINDOWS/System32/OpenSSH:/cygdrive/c/Program Files/PuTTY:/cygdrive/c/Program Files/IcedTeaWeb/WebStart/bin:/cygdrive/c/Program Files/Microsoft VS Code/bin:/cygdrive/c/Program Files/Git/cmd:/cygdrive/c/Users/frede/.cargo/bin:/cygdrive/c/Users/frede/AppData/Local/Programs/DiskuvOCaml/0/bin:/cygdrive/c/Users/frede/AppData/Local/Programs/opam/bin:/cygdrive/c/Users/frede/AppData/Local/Programs/DiskuvOCaml/0/usr/bin:/cygdrive/c/Users/frede/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/texlive/2023/bin/win32:/cygdrive/c/Users/frede/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/users/frede/appdata/local/packages/pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0/localcache/local-packages/python39/scripts:/cygdrive/c/users/frede/.local/bin:/cygdrive/c/texlive/2023/bin/windows:/usr/local/bin:/cygdrive/c/Users/frede/AppData/Local/Coursier/data/bin

There are some . in it which prevents ocamlfind to install (./INSTALL is executed instead of /usr/bin/install). There are some C: which should have been spelt /cygwin/c or something like this. The PATH corruption comes from /cygdrive/c/Users/frede/AppData/Local/opam/opam-init/init.sh (executed from .bash_profile… installed by opam init). It appears /cygdrive/c/Users/frede/AppData/Local/opam/opam-init/variables.sh is updated by opam switch, but fails to use the Cygwin convention about directory paths.

14

This looks like opam 2.2.0 rather than 2.2.1. Can you also run /cygdrive/c/Users/frede/AppData/Local/Programs/opam/bin/opam --version?

The location looks like my legacy packaging of opam 2.2.0. Newer winget install OCaml.opam releases are installed into %LOCALAPPDATA%\Microsoft\WinGet\Links. If you have both opam versions use winget remove Diskuv.opam

While we are at it, for sanity can you clean up paths from ancient DkML 0.x?

  • Remove the directory /cygdrive/c/Users/frede/AppData/Local/Programs/DiskuvOCaml
  • Remove PATH C:\Users\frede\AppData\Local\Programs\DiskuvOCaml\0\bin in Control Panel (System Settings)
  • Remove PATH C:\Users\frede\AppData\Local\Programs\DiskuvOCaml\0\usr\bin in Control Panel (System Settings)

Also, can you dump your opam configuration to check if there are any misconfigured wrap commands and incorrect settings?

opam var
opam option
15

I have installed Opam with the install.ps1 script, executed without admin privileges.

We can read in this script:

DefaultBinDir = If ($IsAdmin) {"$Env:ProgramFiles\opam\bin"} Else {"$Env:L\
OCALAPPDATA\Programs\opam\bin"}

My opam path is consistent with the 2.2.1 install script. opam --version returns 2.2.1.

1 Like
16

I have submit my PATH issue… and a fix about the empty entry has been proposed:

17

I’ve got a hunch about what might have been going on with the .exe.exe part as well, but I need to do a bit more poking!