OCaml for Windows installation confusion

OK, let me summarize the current situation then:

  • As of next month the opam-repository-mingw will stop being updated.
  • But users may be able to add a lower-priority overlay of upstream opam-repository to be able to use any packages which don’t need patching to work on Windows.
  • The official Windows installation method pointed at by the OCaml landing page, is marked with a big ‘deprecated’ warning. To a newcomer, this is a showstopper because they don’t know the above two points.

What I’m trying to say here is that there is no clear, supported, communicated OCaml Platform installation path for newcomers on Windows. Yes, as we know, there are workarounds and other options, but they are not really accessible to newcomers.

2 Likes

I totally agree. Maybe if there was a very clear concise guide that gave step by step instructions for how to get these things setup, but even suggesting overlays and underlays that need to be setup assumes far to much knowledge of the tooling.

One of the big problems is having too many paths and options. Its great that there are options but as a newcomer who just wants to write some code, more options are actually just worse. If one options is faster or slower or whatever it just doesn’t really matter. What does matter is the time from when you get to the ocaml website to when you run and see “Hello World!”

What most newcomers want is a simple guide that takes charge and just says:

This is the way: Do this, then this, then that, then compile and run this example program.
Happy ocamling
PS: there are other options if you care:[link]

Leave the ambiguity and options for the PS and if there isn’t one good way. Just be up front about it.
PS: It’s really great to see that there is progress and support for this kind of stuff, Its so nice to come to a community that is receptive, and actively to trying to improve like this. :+1:

3 Likes

If we are talking about beginners, OCaml does not support Windows yet.

As soon as we start asking people to install cygwin, wsl, or esy, (as opposed to distributing a .msi file) we’ve already lost. Too many things can go wrong and if you spend several hours trying to get these working and are ultimately unsuccessful, the experience can leave a really sour taste in your mouth.

For Windows, I wish ocaml.org should just say:

DON’T DO IT.

PS: If you really really really want to, here’s are your options:

  • cygwin
  • wsl
  • esy
  • etc

Things are expected to improve a lot for OCaml 5.0, but I’m not going to blame anyone if it is not quite there yet by 5.0. It’ll happen when it happens, and I don’t mind waiting another 25 years for beginner friendly Windows support. :grinning:

2 Likes

Some thoughts from a non-Windows user.

  1. I would hope there are a decent number of Windows+OCaml users out there.
  2. And among them, there must be a modest number who are experienced Windows developers.
  3. It seems like, if some number of them were to self-identify, perhaps one could arrange for the developers of the key components of an OCaml developer experience, to give those Windows devs “better support”, and thereby accelerate the process of getting Windows support to where it needs to be?

I’m asking these questions because, quite simply, if there aren’t any experienced Windows devs (if all experienced OCaml users are on Unix variants) then it’s hopeless to expect good Windows support. OCaml isn’t F#, and never will be. It’s got no company of the heft of MSFT behind it, and I doubt it ever will. [and frankly, I hope it never does: corporate “love” seems like the kiss of death, from my 20+ years of enterprise s/w development]

So, uh, maybe if experienced Windows devs (who would be willing to participate in a community to improve developer experience) could raise their hands, that might be useful?

Just a thought.

P.S. Why do I put things this way? Let us suppose that almost all OCaml developers are Unix hackers. Then it’s simply unrealistic to expect the OCaml experience on Windows to be acceptable: you have to have some critical mass of people who care about it, and are able to do something about it.

I’m actually much more comfortable with the idea of some corporate/organization contributing Windows improvements (as unlikely as that sounds). It may be a tall order to get experienced Windows users to contribute, and people should get paid for their work (ideally). Keeping in mind that David and others have made gargantuan improvements already, there are clearly some steps left to a seamless newcomer experience for OCaml on Windows.

If there’s anyone out there who wants to massively help OCaml adoption, improving dev experience for the biggest untapped dev market (less experienced Windows users)–or even paying someone to do the work–is the way to do it.

At LexiFi we have been writing OCaml under Windows for a long time and are quite experienced with it. I know of at least one other large industrial player that uses OCaml on Windows as their main environment cc @keleshev.

If I had to summarize the situation of OCaml on Windows I would say that once you get past the installation of the dev environment (C toolchain + Cygwin), the experience is quite similar to Linux. The compiler, standard library, dune and merlin, all work flawlessly on Windows. OPAM is not 100% ready yet on Windows (we don’t use it) but it has never been easier to work “monorepo” style using dune.

For beginners the main stumbling block is setting up the dev environment (C toolchain + Cygwin). On Linux installing a C toolchain is not needed because the compiler is installed by default, but this is not the case on Windows. As for Cygwin, it is strictly speaking only necessary when building the compiler itself. Once the compiler is installed you are free to never use Cygwin again…

So if you want a Rust-style experience for beginners you need to figure out how to provide a point-and-click installer that does the following things:

  1. installs the C toolchain,
  2. installs Cygwin and builds & installs the OCaml compiler using it
  3. installs dune and merlin

At this point you will be left with a very capable environment for writing OCaml programs using common editors such as VS Code.

Regarding 1. above (the C toolchain), there are two main options. If you want a completely freestanding C toolchain you must use the native MSVC compiler command-line tools, which you can get from Use the Microsoft C++ toolset from the command line | Microsoft Docs

If you are willing to use Cygwin for your day-to-day developing (as opposed to only using it when building the compiler), then you have a second option which is to use the mingw64 compiler. This compiler also produces native Windows binaries, but the advantage is that it is essentially gcc so you get an user experience which you may be used to from Linux. As it runs under Cygwin, this option may appeal to those that prefer to insulate themselves as much as possible from Windows specificities.

It depends what you mean by “on Windows”. If all you want is to develop “on” a Windows machine, yes by all means WSL is a good option. But the binaries you produce are Linux binaries, so they won’t run on Windows outside of the WSL environment.

Only if you insist on using OPAM for your development. It is perfectly feasible to develop on Windows “monorepo” style without using OPAM. The experience may not be as pleasent as it would be on Linux and depending on how much you rely on external libraries it may be more or less convenient, but it works quite well in general.

Cheers,
Nicolas

8 Likes

Thank you. This clarifies a lot. I still think opam is the key. It manages not only OCaml packages but also the compilers themselves. Once opam is working seamlessly on Windows, the experience should be as simple as it is on Unix. E.g.,

C:\> winget install opam
C:\> opam init
C:\> opam switch create ...
C:\> opam install dune utop ...
C:\> dune init ...

And as we saw in the opam site, that (or something similar) is still the plan.

4 Likes

See https://discuss.ocaml.org/t/ann-ocaml-opam-images-for-docker-for-windows for an easy way to get started on Windows using Docker.

Cheers,
Nicolas

2 Likes

Thank you. I think, as avsm said in that thread, that it will be critical to keep Windows support maintained in future. But I don’t think it should be the ultimate end-user (especially beginner) experience. The toolchain offered should be as close to native as possible, not force a new dependency i.e. Docker.

1 Like

I think you should recognise that it is an unrealistic expectation that the OCaml ecosystem can overnight have a fully maintained, regularly released Windows toolchain arriving out of every single OCaml compiler and platform tool release.

Getting to this end goal requires us to make steady, iterative improvements that both mechanise the release process (something notoriously difficult on Windows), and match the velocity of changes in a *nix-centred ecosystem (start testing new opam packages to at least stop more Windows incompatibilities creeping in, as the core compiler team has done for years, and then run health-checks on existing packages).

The Windows Docker images here represent:

  • a reproducible(ish) pipeline that builds an opam-repo/opam across all supported Windows toolchain combinations.
  • a composable pipeline, since you can stack more application builds over the base images, and swap base images to test out new toolchains fairly easily.
  • an integration point with modern development tools like VS Code, which supports seamless in-container development.

If you want to help get to our endgoal of a fully supported Windows/OCaml/opam ecosystem, we need more people spending time on trying out the containers and see how well it does work for them in Windows setups, and reporting/fixing incompatibilities in opam-repository (as @fdopen did with his opam-repository for a long time).

Once the testing pipeline settles down, we need to do the next set of steps to further improve support: roll this out to the opam-repository CI, and generate binary installers for Windows, and so on. But we’re not going to get there by hypothesising about the perfect beginners experience in forum threads :slight_smile:

3 Likes

I fully agree with everything you said :slight_smile: That’s why I said, ‘it will be critical to keep Windows support maintained in future’. I understand that it’s a critical piece of the OCaml Windows infrastructure. And it will help us to solve the ‘last mile problem’, i.e. delivering the perfect onboarding experience. The reason I started this thread is that I believe solving that will give OCaml a good adoption boost–especially once multicore is shipped and there is renewed interest and scrutiny.

2 Likes

Two weeks ago, I asked if there were experienced Windows developers using OCaml on Windows. One person/group ( @nojb at LexiFi ) raised their hand. It makes one wonder if this means there just isn’t a sufficient population of senior Windows devs out there, to make “a great native Windows experience” an achievable goal.

To state the obvious, asking on this forum isn’t a representative sample. @dra27 and @djs55 and @keleshev and many others using Windows and OCaml happily didn’t raise their hands. It’s an entirely achievable goal, and the hard work of several developers is incrementally getting us there. We just announced a massive step forward today…

3 Likes

Sorry I missed this post. I’m a happy Windows OCaml user too: I’ve been using OCaml for a long time and Windows specifically for about 5 years or so. I’m a heavy (and extremely grateful) user of @fdopen’s opam-repository-mingw which I’ve used as a base on top of which I’ve layered my own packages and pins. I’m really looking forward to switching my main Windows project moby/vpnkit to a more “vendoring” style with ocamllabs/opam-monorepo so that new developers can git clone and then dune build everything together. I’m also impressed with the available scalable I/O libraries for Windows. I’m currently using libuv via fdopen/uwt and am experimenting with aantron/luv, both of which are excellent.
Perhaps I’m biased since I work on Docker :slight_smile: but I’m really pleased with the new OCaml on Windows base images. I’m planning on using these in CI, so I can docker build artefacts for release.
I completely agree that the ideal end state will be an ocaml.msi which installs all the tools, and where everything just works out of the box. I think we’re moving in the right direction and we’ll get there … and I will try to help where I can!

4 Likes

I would like to learn more about this native Windows experience that that we should strive for. What are examples of languages doing this right? Except for Microsoft-developed languages, of course.

I spend some fraction of my time using C# with Visual Studio and TFS, and it is an experience that I can’t recommend. A good debugger is the only excellent part of it, the rest is dreadful, in my opinion.

I also don’t follow any aversion towards Cygwin. As soon as you want to use Git on Windows, you have to install Git Bash which is based on Mingw, so you quickly end up using some sort of Unix. Same with Python, Node, Ruby. It seems that some kind of Unix shell is a strong prerequisite for almost all kinds of development on Windows nowadays.

I have never (ok, barely ever) had any problems using @fdopen’s excellent installer and then using opam from there. At work, we are using a monorepo approach similar to what @nojb is describing, so it is a bit more involved. The idea is very similar to what @avsm described in his 2019 presentation at the OCaml Workshop (can’t quite find the recording), just more hand rolled.

Together with with .NET FFI system csml from LexiFi you can build Windows desktop applications that are undistinguishable from native Windows ones. The questions is—who wants Windows desktop applications? At SimCorp, we are currently migrating our nice Windows applications (also ones backed by OCaml) to cloud-based offerings.

I agree that the messaging around OCaml on Windows is not very clear. Going to ocaml.org for installation instructions only to find that “this repository is deprecated” is not the best experience.

I was told (but haven’t tried it first-hand) that Rust is a good example.

Cheers,
Nicolas

1 Like

Let’s take a look at installing Node.js on Windows. First you go to the download page: Download | Node.js

Then you download the Windows installer (.msi) and run it.

Finally, to verify, you open the command prompt (or PowerShell) and run node -v and npm -v to check the installed versions.

No mention of Unix shells anywhere in this process. I think this is a good beginner experience. Of course it probably took a lot of work to get Node to this point. But it is an example of the end goal.

1 Like

@yawaramin I think we are quite close to this. It takes a bit of searching to find the link to OCaml64.exe, which is a regular Windows installer, like the rest of them, and by the end of it you get a desktop shortcut which opens a shell where you can type ocaml, opam, dune, etc. Do people actually use command prompt (or PowerShell) with Node?

Good to hear there is a regular Windows installer already.

Yes, a lot or most Windows developers use the command prompt i.e. cmd.exe or PowerShell because usually all they need to do is run some server with e.g. npm run start command or npm test and then get into the dev iteration loop. They don’t need sophisticated Unix shell capabilities.

2 Likes

Yes. All of the main interpreted languages (node, python, ruby) are easily available in Windows. The result is that you can run them from the command prompt if you want, you can create full desktop applications with a GUI, run and debug from the IDE, etc. Of course, it’s easier with interpreted languages because you don’t need to make compilation work. Windows doesn’t even ship with an easily available C compiler, which compiled languages often require as part of their compilation process. But mingw can be thrown in there pretty easily.

git bash is actually the exception to the rule. Despite being an application (and therefore similar to interpreted languages), git is a mess of scripts often written in bash rather than a full application like mercurial. This is why a simple port to Windows was never possible. Regardless, it appears to me that most Windows users prefer to use a GUI such as Visual Studio Code to manage git (which is the smart way to go anyway since individual file management via the git command line is a pain).

Despite the fact that porting compiled languages to Windows takes more effort, it appears that Haskell now works natively on Windows, as does Rust, as mentioned above. Notice that they recommend installing through the best currently available Windows package manager, which is Chocolatey.

Cygwin is pretty awful. It’s very slow; it’s hard to track where it picks up dependencies; it’s completely foreign to Windows users; and when you want to install new packages, you need to shut it down and use the awful cygwin installer (I think there are some workarounds now, but it’s still not a great experience). The reality is that Windows users aren’t familiar with the world of sh and its countless utilities, and they don’t need to be – the Windows world works mostly via GUIs and IDEs that activate required functionalities. Few people in the Windows world want to worry about cygwin dlls and applications that require them, or which version of mingw to use to enable building fully-capable Windows apps on cygwin. This knowledge is usually reserved for a few technical people in the organization who set up the scripts for the GUIs to works, while other programmers just focus on coding.

3 Likes