Weird path errors building stuff for Windows in Github CI

Learning
1

I’m kind of banging my head against a wall here. I’m trying to build an OCaml program for Windows in Github CI. Here is the failing job. It’s failing to build linksem with this error:

# ocamlfind remove -destdir D:\a\sail\sail\_opam\lib linksem
Warning: 0mocamlfind: [WARNING] No such file: D:\a\sail\sail\_opam\lib\linksem\META
# ocamlfind install -destdir D:\a\sail\sail\_opam\lib -patch-version "3e5a0af" linksem META
# ocamlfind: Bad configuration: Cannot mkdir D:asailsail_opamlib\linksem because a path component does not exist or is not a directory
# make[1]: *** [no_ocamlbuild.mk:75: D:\a\sail\sail\_opam\lib/linksem/META] Error 2
# make[1]: Leaving directory '/cygdrive/d/a/sail/sail/_opam/.opam-switch/build/linksem.0.8/src'
# make: *** [Makefile:10: install] Error 2

As you can see, the path seems to have been mangled by Windows’ insistence on backslashes. I guess at some point they were interpreted as escapes. But I can’t find where that could be at all. Especially because the previous [WARNING] No such file: gets it right and as you can see -destdir is the same in both cases.

I looked through the the ocamlfind code but can’t see any obvious mistakes or differences between the paths to [WARNING] No such file and Cannot mkdir.

The weirdest thing is that it works fine on my computer! I tried the exact same version of OCaml, and it’s the same version of linksem. This is the worst kind of CI failure. Does anyone have any ideas?

2

It’s a bit difficult to diagnose without knowing what is invoking what, where and in which environment. The things “you see” (are being printed) may not be what the tools are actually given.

Can you provide a more precise “stack trace” (opam invokes this makefile that invokes this bla bla bla) to the failing invocation. Including the one that succeeds on your computer. ?

3

The things “you see” (are being printed) may not be what the tools are actually given.

I think they are because it prints the commands that are run:

# ocamlfind remove -destdir D:\a\sail\sail\_opam\lib linksem

and

# ocamlfind install -destdir D:\a\sail\sail\_opam\lib -patch-version "3e5a0af" linksem META

That comes from this Makefile where -destdir is passed identically:

	-ocamlfind remove -destdir "$(INSTALLDIR)" linksem
	ocamlfind install -destdir "$(INSTALLDIR)" -patch-version "$(LINKSEMVERSION)" linksem META

Can you provide a more precise “stack trace” (opam invokes this makefile that invokes this bla bla bla) to the failing invocation. Including the one that succeeds on your computer. ?

It is essentially:

opam install --deps-only .

I have to set the LEMLIB environment variable but it appears that it is being set correctly on Github.

This is really weird. I’ll try simplifying the script to just opam install linksem so there are fewer steps.

I’m not familiar enough with opam to know how it gets from that to the Makefile.

Also how do I debug this? Can I fork linksem and add debugging to the Makefile? If I do that how do I tell opam to install from my fork?

How would I get it to use a forked version of ocamlfind with more debugging?

4

Yes. Fork the repo and then do

opam pin add linksem git+https://url/to/repo.git#branch

(same for ocamlfind)

1 Like
5

Damn I tried simplifying to just opam install linksem and now I get a different error:

x86_64-w64-mingw32-gcc: fatal error: '-fuse-linker-plugin', but cyglto_plugin.dll not found

StackOverflow suggests this is due to having a mix of gcc-c++ and gcc-core cygwin versions but how? This is all it’s doing.

6

Ok that error seems to have gone away for some reason. Maybe a glitch. Anyway I successfully patched ocamlfind to print argv right at the start of main(). This is super weird!

First it runs this:

# ocamlfind remove -destdir D:\a\sail\sail\_opam\lib linksem
# OCAMLFIND: argv is [
# D:\a\sail\sail\_opam\bin\ocamlfind.exe
# remove
# -destdir
# D:\a\sail\sail\_opam\lib
# linksem
# ]
Warning: 0mocamlfind: [WARNING] No such file: D:\a\sail\sail\_opam\lib\linksem\META

Totally fine. Then it runs a very similar command.

# ocamlfind install -destdir D:\a\sail\sail\_opam\lib -patch-version "2a2bb1f" linksem META
# OCAMLFIND: argv is [
# D:\a\sail\sail\_opam\bin\ocamlfind.exe
# install
# -destdir
# D:asailsail_opamlib
# -patch-version
# 2a2bb1f
# linksem
# META
# ]

Mangled -destdir! Wtf. The only difference I can see between those commands is the - at the start which apparently just means Make ignores errors. Hmm.

7

Ugh well this sucks but it was some path issue in the Makefile that has been fixed in master but not released to opam. I still don’t quite understand why this issue only affected the second command - even before the patch they were called in the same way. I guess it’s just one of Make’s many many wtfs.

And as for why it worked on my computer… I checked opam pin list and it turns out I had already pinned linksem to the latest version. I guess I figured it out when I started working on this a few months ago and then totally forgot. Debugging CI sucks so much. If anyone manages to solve that (in a way that isn’t just “we support docker!”) they’ll make a fortune!

1 Like
8

I’ll be that person: at this point you could use a better build system than ocamlbuild, like Dune, or BRZO.

9

Yeah definitely. Not my project though. I think the author did have a go recently but there was some issue that made it non-trivial.

10

@Rucikir I do think brzo is an extremely cool zero-config build system for exploratory programming and booting projects but, as written on the page you linked to, it’s not suitable for building distributions.

And each year I tell myself this is the year I will officially release brzo and the year passes by too quickly :–(

2 Likes