[ANN] Diskuv OCaml 1.1.0 - Windows, winget, setup-dkml

Ecosystem
,
1

I’m pleased to announce the release 1.1.0 of DKML (Diskuv OCaml) at https://github.com/diskuv/dkml-installer-ocaml/releases/tag/v1.1.0_r2. As usual the DKML distribution supports the Visual Studio compiler on Windows. After the 90-minute installer finishes you will have a working environment where dune build and utop work immediately. The installed OCaml language server works in VS Code using the installed dkml switch with a single VS Code option change (see Full docs). Creating a new local opam switch with an ocaml-system compiler takes about 90 seconds with dkml init. Full docs are at Getting Started with Diskuv OCaml — Diskuv OCaml 1.1.0 documentation.

Scroll below for related announcements about setup-dkml (CI for MSVC) and winget.

There are many new features and changes, so please browse the following release notes:

Quick Links:
Installer https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.1.0_r2/setup-diskuv-ocaml-windows_x86_64-1.1.0.exe
Uninstaller https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.1.0_r2/uninstall-diskuv-ocaml-windows_x86_64-1.1.0.exe

Upgrading from 1.0.1?

  1. Uninstall the old version first with the uninstaller above!
  2. After uninstalling the old version, run the following in PowerShell:
    if (Test-Path "$env:LOCALAPPDATA\opam\playground") { Remove-Item -Path "$env:LOCALAPPDATA\opam\playground" -Force -Recurse }
  3. Exit Visual Studio Code and any utop or ocaml sessions. Then use the installer above.
  4. After installing the new version, run the following in PowerShell in each directory that has a local opam switch to upgrade to OCaml 4.14.0 and all the other package versions that come with DKML 1.1.0:
    # Sometimes `dkml init` can fail, but you are OK as long as you see:
# ...  upgrade   ocaml-system                       4.12.1 to 4.14.0
dkml init

opam upgrade
opam install . --deps-only --with-test

Cautions:

  • Do not use this distribution if you have a space in your username (ex. C:\Users\Jane Smith). Sorry, but the broader OCaml ecosystem does not yet consistently support spaces in directories.
  • Your Windows anti-virus may play havoc with the installer. If possible, temporarily disable your anti-virus (the “realtime protection”, “exploit protection” and/or “malware protection” options). Some anti-virus products include a button to temporarily disable AV protection for two hours; do that. If you forget and the installer fails, you will need to disable AV protection, run the uninstaller, and then rerun the installer!

New features:

  • The system OCaml is 4.14.0 (was 4.12.1)
  • The system includes ocamlnat, the experimental native toplevel. It should be run using with-dkml ocamlnat so native code is compiled with Visual Studio.
  • Add odoc 2.1.0 to user PATH, to align with the OCaml Platform.
  • Relocatable native binaries are installed rather than compiled into place. Installations should be quicker, which is a pre-requisite for winget install (pending!) on Windows.
  • Add opam global variable sys-pkg-manager-cmd-msys2 for future compatibility
    with opam 2.2 depext support of MSYS2
  • The opam dkml init command is now dkml init. The dkml executable is precompiled and shaves ~20 minutes of installation time.

New security:

  • (Advanced; experimental) If you are behind a corporate firewall that uses man-in-the-middle (MITM) TLS proxying, you can install your corporate CA chain so DKML, in particular MSYS2, does not reject connections. Only persons with write access to $env:ProgramData\DiskuvOCaml\conf\unixutils.sexp will be able to define the allowed MITM TLS chain; you may need access from your corporate Administrator. An example unixutils.sexp is:

    (
    (trust_anchors ("C:\\conf\\my.pem" "D:\\conf\\my.cer"))
)

    You specify one or more .pem or .cer CA files, making sure to use two backslashes to escape your paths. Your Administrator may have already placed the CA files on your machine; otherwise use the guide at FAQ - MSYS2 to copy them from your web browsers.

Not so good problems:

  • Known bug #21 To install the OCaml language server in a new switch you will need to first do opam pin remove fiber omd stdune dyn ordering --no-action before doing opam install ocaml-lsp-server.
  • Many opam packages do not work with the MSVC compiler or with Windows. It will take a long time (months, years) to substantially improve Windows coverage. When you do find a package that fails to compile on Windows, please file an issue with whoever owns the package expressing interest in the package working on Windows. Please be patient: some package owners may want to see several people express interest before deciding the extra work is justified.

Breaking changes:

  • Cross-compiling on macOS with dkml-base-compiler now requires you to be explicit which CPU architecture you are targeting. Before using dune -x darwin_arm64 would always cross-compile both Intel and Silicon. Now Silicon developers need to use dune -x darwin_x86_64 and Intel developers need to use dune -x darwin_arm64. The change was necessary to not rely on the presence of optional Rosetta2 translation. Since this cross-compiling feature is little used, it does not warrant a breaking version bump.

Documentation changes:

Bug fixes:

  • The dune.exe shim uses a cache containing the expensive-to-compute MSVC environment settings. A race condition populating the cache has been fixed.

  • Repetitive opam repository updates, a source of slowness, were eliminated during installation.

  • ocaml-crunch upgraded (pinned) from 3.2.0 to 3.3.1 to fix Windows/Unix path inconsistency and handling of CRLF. That and other changed package versions are:

    Package Old Version New Version
    dune 2.9.3 3.6.2
    ocaml-crunch 3.2.0 3.3.1
    cmdliner 1.0.4 1.1.1
    uuidm 0.9.7 0.9.8
    ptime 0.8.6 1.1.0
    sexplib v0.14.0 v0.15.1
    lsp 1.9.0 1.10.3
    ocaml-lsp-server 1.9.0 1.10.3
    jsonrpc 1.9.0 1.10.3
    odoc 2.1.0 2.2.0
    odoc-parser 0.9.0 1.0.1
    stdio v0.14.0 v0.15.0
    base v0.14.2 v0.15.1
    mdx 2.0.0 2.1.0
    ocamlformat 0.19.0 0.23.0
    ocamlformat-rpc 0.19.0 not pinned
    ocamlformat-rpc-lib 0.19.0 0.23.0

  • The Visual Studio installation cleans up aborted previous installations.

  • The Visual Studio installation, on failure, writes error logs to a location that isn’t immediately erased.

Component changes:

  • opam.exe is compiled directly from the opam master branch; no patches! There is still a shim but that shim just sets up environment variables and delegates to the authoritative (unpatched) opam. There is one patch for opam on top of the opam master branch (opam 2.2) dated 2022-12-21.
  • MSYS2 setup program is bundled inside the installer to lessen download TLS problems when a proxy is present (common with corporate/school Windows PCs). Resolves Installer fails at curl due to self-signed certificate · Issue #19 · diskuv/dkml-installer-ocaml · GitHub

Reproducibility features:

  • Packages promoted to central Opam repository:
    • dkml-runtime-common
    • dkml-runtime-distribution
    • dkml-component-opam
  • Introduce simple spec for which package+versions are installed and/or compiled as part of the DKML distribution (in dkml-runtime-distribution). Eventually it will become authoritative.
  • Introduce dkml-component-desktop which does CI for changes to that spec (aka. testing new package versions for Windows using MSVC), and builds all relocatable binaries like dune and ocp-indent used in the Windows installer. It is not yet in the central Opam repository.
  • During installation any CAMLLIB environment variable (in addition to OCAMLLIB which was already checked) is renamed to deconflict with a new OCaml installation. Among other things, this provides an upgrade from CamlLight to OCaml. Unbound module Stdlib · Issue #13 · diskuv/dkml-installer-ocaml · GitHub

Usability tweaks:

  • When building DKML packages like dkml-base-compiler, limit dump of Opam logs used for troubleshooting to 4 hours

Feature flags:

  • Enable --enable-imprecise-c99-float-ops during OCaml compilation by setting

    (
  (feature_flag_imprecise_c99_float_ops)
)

    in $env:ProgramData\DiskuvOCaml\conf\ocamlcompiler.sexp. This is sometimes needed inside virtual machines like Vagrant

Licensing:

  • Diskuv OCaml is fully open-source, and is targeted for pure OCaml development. Commercial tools and support are available from Diskuv for mixed OCaml/C development; however, Diskuv OCaml only has limited support for mixed OCaml/C. For example, the ctypes opam package has been patched to work with Visual Studio but is out-dated. Contact support AT diskuv.com if you need OCaml/C development.

Internal changes:

  • with-dkml.exe is now configured as a opam wrapper relative to the installation directory ($DiskuvOCamlHome) rather than the tools opam switch. That change decouples your new switches (aka. dkml init) from another opam switch.
7 Likes
2

I don’t use Windows for development, but I hope this is a typo and you meant 90 seconds, instead of 90 minutes?

3

Definitely not a typo. Installing Visual Studio (the C compiler and Windows SDK) by itself takes 10 minutes, and that is just one of many components installed (MSYS2, opam init, OCaml compiler, etc.). 90 seconds will never be a goal.

2 Likes
4

winget

This is the first release of DKML in winget, the (new) Windows Package Manager.

Windows Package Manager winget command-line tool is bundled with Windows 11 and modern versions of Windows 10 by default as the App Installer.
If you are running an earlier version of Windows and the App Installer is not
installed, you can get App Installer from the Microsoft Store.
If it’s already installed, make sure it is updated with the latest version.

Once you have winget, uninstall any existing DKML using the uninstall link (see the main announcement) and then type the following for the 90-minute install:

winget install Diskuv.OCaml

You will then be able to start hacking in a new PowerShell window at Install is done! What next?

3 Likes
5

setup-dkml 1.2.0

How to check if you are using setup-dkml? You have a ci/setup-dkml folder.

The new version of setup-dkml CI goes along with the DKML 1.1.0 release (yes, it would be nice if the version numbers were in sync). You will get CI for Windows MSVC with OCaml 4.14.0 (among other things) when you upgrade.

You should check ci/setup-dkml/*.md to see if there is any upgrade documentation. If not, you can upgrade with:

opam update
opam upgrade dkml-workflows
opam exec -- generate-setup-dkml-scaffold
opam exec -- dune build '@gen-dkml' --auto-promote
2 Likes
6

It would be nice to understand exactly where the time goes. Linux users compile from scratch and yet don’t take anywhere near as long. Aside from VS, what else is causing the major delays? Is it msys2? Is it the msys2 translation layer slowing everything down?