opam-cross-* are seriously lagging behind the official opam repository and fdopen’s opam-windows, not least because importing packages by hand is a lot of work.
I suppose at least a semi-automated process could help those repos grow and stay in sync with the upstream much faster.
only dune is supported by automatic build command adjustment
it cannot handle cases when both native and cross-version of a dependency are needed
However:
For simple packages that use dune exclusively, it’s completely automated. I’ve ported bigstreamaf and angstrom to test it, and cross-versions built just fine from its output, no editing was needed.
It automatically converts dependencies from foo to too-$toolchain and removes dependencies and build steps only needed for with-test and with-doc.
That’s a great initiative! Here are a couple of thoughts:
For dune-based packages, things are indeed pretty straight-forward. Finding out which dependencies need to be ported as cross-dependency is indeed the part that’s hard to automatize
For other build systems, it’s less clear to me how to automatize. Maybe others have some thoughts about it.
The CI system on opam-cross-windows is pretty good at building from scratch and failing if some deps are missing so trial and error there can be a great tool.
Once solved for one cross situation, the problem of cross-dependencies should be exactly the same for all other cross environment (android, iOS)
I haven’t looked at the tool very closely yet but I’d say a first improvement would be to be able to track cross-dependencies resolution and generate new version of the package using them and/or generate other cross-compiled packages using them.
In my imagination, it will work like this: initial package import will be computer-aided but with a human in the leading role. However, subsequent updates can be made by robotic pull requests, since packages rarely make big changes to their dependencies.
At the point when most packages from opam-repository have cross-versions, importing new packages will be much simpler because their dependencies will likely be already available.
I’m not sure if I understand the premise of dependabot. Why would anyone hardcode specific dependency versions? Maybe it makes sense in certain ecosystems that suffer from never-ending ecological disasters…
In any case, most opam packages don’t have a constraint on the upper versions of their dependencies. Can dependabot use custom tracking rules to check for presense of a newer version in the repo?
My thought was much simpler actually: track the commits in opam-repository, run recently changed files through Dismas and send pull requests to opam-cross-*
It’s common practice nowadays to use semantic versioning and have lockfiles for reproducible builds. Dependabot updates semantic version ranges and lockfiles. See e.g.
This is really nice! I was thinking about implementing a tool like this back when I started opam-cross-*, but the ecosystem didn’t seem quite ready for it, and I ended up pushing for cross-compilation support in ocamlfind, topkg, etc instead. I’m really happy to see that dune now allowss turnkey conversion of normal packages to cross packages.
I’m not sure if the ecosystem is completely ready for it now. I’m pretty sure if/when this takes off, many packages will need fixes to be friendly to unattended porting.
Still, from my experience porting 20+ packages to opam-cross-windows (an experience I hope to never repeat, hence the tool), only a couple of them needed editing that requires understanding of the build system, the rest was more or less mindless, so it gives me hope.
I wonder if we should add a dune-windows dependency to all packages that use dune just in case they need any dune libs, or we should limit it to those that explicitly depend on dune-configurator etc. and send patches to the package maintainer to make those dependencies explicit.
The ecosystem is never completely ready for something until you drag it there, kicking and screaming; or, more often, tired and indifferent. Though that figure of speech is really overly dramatic here, I found that most people working on OCaml projects are happy to assist.
In other words, the only way to get the (even well-behaved dune subset of) ecosystem ready is to chip away until automated cross-compilation porting becomes routine and the idea of making it more straightforward becomes natural. When I started you couldn’t even use ocamlfind for cross-builds, now dune makes it pretty much a standard feature! On the flipside, you have to be in for a long game.
