# [ANN] OCaml+Opam Images for Docker for Windows

Ecosystem
, , ,
1

[ANN] OCaml+opam Images for Docker for Windows

Hello all,

I’m glad to announce the availability of OCaml and opam native Windows Container images for Docker for Windows. This is the result of my hard work at Tarides, with precious help from @dra27, @talex5, @avsm, and the rest of the team.

They can be found under the ocaml/opam repository in the Docker Hub. Try them with Docker for Windows! Be sure to switch Docker to Native Windows Containers.

docker run -it ocaml/opam:windows-mingw
docker run -it ocaml/opam:windows-msvc

We provide images for the mingw-w64 (from OCaml 4.02 to 4.12) and the MSVC (from OCaml 4.06 to 4.12) ports. They are based on each release of Windows 10 amd64 currently supported by Microsoft on the Docker Hub. The images use opam 2.0, and we plan to update to opam 2.1 when it’s released. The images also ship a Cygwin installation, Git for Windows, and the winget package manager.

We use @fdopen’s OCaml for Windows distribution and opam-repository fork. As it is getting deprecated at the end of August 2021, we’ll transition to opam 2.1 and the standard opam-repository when that happens.

In order to get the correct environment for any RUN command involving OCaml or opam, prefix the command with

  • ocaml-env exec --64 -- if based on mingw-w64; or
  • ocaml-env exec --64 --ms=vs2019 -- if based on MSVC.

The images are built at https://base-images.ocamllabs.io/, using an OCurrent pipeline that builds Docker images. You can rebuild them yourself using the OCluster set of tools that I have ported to Windows.

We provide a comprehensive set of tags (replace port with either mingw or msvc):

  • windows-port: the latest version of OCaml for each Windows version;
  • windows-port-winver: the latest version of OCaml for Windows 10 winver;
  • windows-port-ocaml-mlver: OCaml version mlver for each Windows version;
  • windows-port-winver-ocaml-mlver: OCaml version mlver for Window 10 winver.

When the Windows version is not specified in the tag, the image is a multiarch image that will work on every supported version of Windows 10. Docker automatically selects the appropriate one based on the host version.

We will be using these images in the upcoming ocaml-ci and opam-repo-ci for Windows.

Further work on these include the transition to opam 2.1, and we’ll provide the Cygwin port of OCaml when it’s fixed upstream and available in the Cygwin package repository.

Happy hacking!

– Antonin

28 Likes
2

For those curious about the history of this work, @Rucikir began this journey back in 2020 with an internship with @Zimm_i48 and @yurug. That internship helped us discover all the intricacies of what was required for Windows support, and then @Rucikir joined Tarides and has subsequently been hacking away with @dra27 and @talex5 at OCaml Labs (with advice on @djs55 at Docker) during these pandemic months and introducing Windows support into the core OCaml CI stack. This includes a number of fixes to the OCaml compiler itself!

I can’t think of a better showcase for the value of open source development – this has been a wonderful collaborative effort across multiple organisations and countries, and congratulations to @Rucikir on persevering through all the software layers and details to shipping this in the production infrastructure for ocaml.org. And of course, huge thanks to @fdopen for his mingw-opam-repository and @dra27 and @nojb and the core OCaml team for such excellent ongoing support from the compiler itself.

As a community, these regularly built Windows images will form the basis for full opam/OCaml support of Windows. The work being deployed here makes it possible to not only support Windows on a snapshot of our ecosystem, but also to keep Windows support working on an ongoing basis!

18 Likes
3

I just tried it – nice!

Thanks a lot for this, I can only imagine it must have been painstaking work to get everything working.

Cross-referencing https://discuss.ocaml.org/t/ocaml-for-windows-installation-confusion/ I think these docker images could/should be recommended as THE way to get started under Windows.

Cheers,
Nicolas

3 Likes
4

If anyone gets the chance to try out the VS Code container extension with these Windows images, that might be a fine thing to add to the recommendation list.

The base images are ~8-10GB in size, so this isn’t a lightweight option, but it is one that should be fairly well integrated with VSCode/Windows 10 for those users who do need to generate Windows executables without cross-compilation.

3 Likes
5

Thanks everyone for making this happen! This was a really huge step. And I now have a relatively painless but lengthy Windows install procedure on top of your OCaml+Opam images.

I’ll try to publish it late next week or the week after, but in summary after you:

  1. Git clone a project and
  2. Run a PowerShell script

the following will take place automatically over ~2 hours (too long!):

  1. Install Cygwin and MSYS2 and MSBuild (Visual Studio Build Tools)
  2. Download OCaml+Opam images without requiring Docker on your machine
  3. Recompile the Cygwin-linked opam.exe from the OCaml+Opam image into a pure Windows opam.exe
  4. Creates an Opam root with a global 4.12 switch in the user’s $env:USERPROFILE home directory, configured/compiled for the MSBuild/MSVC toolchain and lightly patched for use with MSYS2
  5. Installs ocaml*, dune, opam, utop and ocamllsp in a separate user-specific directory similar to other Windows programs, and makes them available on the user’s PATH
  6. Creates an external Opam switch for the local project

That will:

  • give you access to OCaml, Dune, etc. from any Command Prompt or PowerShell
  • let your local OCaml project work out of the box with the VS Code OCaml Platform extension (pending PR https://github.com/ocamllabs/vscode-ocaml-platform/pull/682) in both Windows and WSL2 Ubuntu
  • let you drop into a MSYS2 shell from any directory to do dune build, opam install and any other OCaml commands that require access to Unix tools or a C compiler toolchain

The biggest rough edge in this OCaml distribution (so far) is the use of copying rather than symlinks to ensure maximum Windows compatibility; patches to Opam will be submitted soon. And I haven’t had the time to tease out how dependent the distribution is on the deprecated fdopen-mingw repository.

@Rucikir @avsm Key question before I release it: Are we willing to add MSYS2 builds to OCaml CI? It is in no one’s interest to release a MSYS2 backed OCaml distribution if we are committed long-term to Cygwin! My main reason for leaning on MSYS2 is because of Cygwin’s frequent instability; specifically a) Cygwin just hangs for no apparent reason if you use it heavily for builds (FAQ 4.32) and b) the popup “shared region version mismatch” of FAQ 4.20 can swarm you and sometimes requires opening Task Manager / logging out to fix. I’ve had both happen multiple times in the past couple weeks.
(MSYS2 minimizes “a” and “b” by partially redistributing MinGW Windows executables that don’t use a lock-heavy shared DLL.)

Thanks! Jonah

7 Likes
6

@jbeckford this looks excellent.

If you have time for it would you also be able to add or link to instructions on how to install git and where to get windows VMs.

A long time ago I managed, under the strict supervision of @dra27, to get myself an (opamless) vm setup to test some of my software but I suspect it has rotten badly.

Having easy steps for us non (or reluctant) Windows users to test/build OCaml software on the platform will also help supporting it better.

7

Testing + documenting the VM install may take a bit of time but it sounds worthwhile. Will look into it.

1 Like
8

@dbuenzli I’ll send out an [ANN] for my Windows distribution in the next day or so but here are the manual (not Vagrant) Windows VM instructions: Windows 10 on macOS/Linux with VirtualBox. The VM was a great suggestion; using a VM uncovered many bugs I had not noticed on my own machines. Thanks!

3 Likes
9

Historically, the issue here is that MSYS2 is a fork of Cygwin, so it’s always felt that this about maintaining Cygwin and MSYS2 (rather than Cygwin or MSYS2). OCaml itself is not likely to have official MSYS2 support any time soon (maintaining the existing support for mingw-w64 - MSVC - Cygwin is already quite a burden on CI!), but I certainly (warmly) welcome patches for MSYS2 support in opam.

10

I do think it is Cygwin or MSYS rather than Cygwin and MSYS. Simply put, there is no reason to use Cygwin at all. I realize it will take a bit of time to establish that fact, but the distribution I released is one proof we don’t need Cygwin. A long-term concern I have is that the broader build community has already decided to invest in MSYS2 rather than Cygwin for Windows build tools (ex. Git for Windows runs on MSYS2; CMake has officially supported generators for MSYS2; vcpkg uses MSYS2 for Windows builds, etc.). The contrast with OCaml CI quite literally increasing its investment in Cygwin and the much larger blast radius on the rest of the OCaml community is worrying but not the end of the world. And I’m avoiding discussion of the impact of WSL2 stealing away the Cygwin userbase. Anyway, food for thought.

Thanks! Already have a few I’ll be sending over when I get some spare cycles.

2 Likes
11

Sorry if this was discussed above, but: have you tried/been able to build OCaml from source with one or both of mingw64/msvc toolchains on MSYS2?

Cheers,
Nicolas

12

If by “OCaml” you mean ocamlc/ocamlrun/ocaml* then yes: https://gitlab.com/diskuv/diskuv-ocaml/-/tree/main/etc/opam-repositories/diskuv-opam-repo/packages/ocaml-variants/ocaml-variants.4.12.0+msvc64+msys2 . That is used by the Diskuv OCaml distribution (sorry for mixing threads; it is in a recent [ANN] thread). Edit: This was a variant build using opam.exe to do all the compiling, not from the starting point of me personally doing a git clone of the ocaml/ocaml repository (you mentioned “from source”). I haven’t tried that, nor do I want to!

If by “OCaml” you mean OCaml packages in general then also yes. Many OCaml packages compile (ex. core_kernel which has many dependencies), a few of which needed patching.

To be clear, I am not claiming there are no problems getting packages to compile with MSVC+MSYS2. Patches are still required (although not many), just like there are patches needed for Cygwin. The patches for Cygwin and MSYS2 are often different though!

1 Like
13

Anyone successfully compile latest unison with this container, specifically windows-msvc-ocsml build? Appreciate your hints on how to accomplish this process.

14

I think it’s worth starting a new thread specifically for Unison.

15

Done here