OCaml 4.12.0, first release candidate

Ecosystem
1

The release of OCaml 4.12.0 is expected next week. We have created a release
candidate that you can test. Most opam packages should work with this release
candidate (without the need for an alpha repository).

Compared to the last beta, this new release only contains one fix for Solaris
and illumos.

If you find any bugs, please report them here:
Issues · ocaml/ocaml · GitHub

Happy hacking,

– Florian Angeletti for the OCaml team.

Installation instructions

The base compiler can be installed as an opam switch with the following commands

opam update
opam switch create 4.12.0~rc1 --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

If you want to tweak the configuration of the compiler, you can pick configuration options with

opam update
opam switch create <switch_name> --packages=ocaml-variants.4.12.0~rc1+options,<option_list> --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

where <option_list> is a comma separated list of ocaml-option-* packages. For
instance, for a flambda and afl enabled switch:

opam switch create 4.12.0~rc1+flambda+afl --packages=ocaml-variants.4.12.0~rc1+options,ocaml-option-flambda,ocaml-option-afl --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

All available options can be listed with opam search ocaml-option.

The source code is available at these addresses:

https://github.com/ocaml/ocaml/archive/4.12.0-rc1.tar.gz
https://caml.inria.fr/pub/distrib/ocaml-4.12/ocaml-4.12.0~rc1.tar.gz

9 Likes
2

Previous topics from @octachron on the 4.12.0 release:

(I don’t think we ever had so many preparation releases for new OCaml versions. Maybe the change is due to the ever-increasing integration with opam-repository testing led by @kit-ty-kate, where it is convenient to have fixed compiler pre-releases to test against?)

I recently worked on cleaning up the 4.12 changelog, which led me to highlight a bunch of changes that I think could be most impactful to user:

Supported platforms (highlights):

  • #9699: add support for iOS and macOS on ARM 64 bits
    (Eduardo Rafael, review by Xavier Leroy, Nicolás Ojeda Bär
    and Anil Madhavapeddy, additional testing by Michael Schmidt)

Standard library (highlights):

  • #9797: Add Sys.mkdir and Sys.rmdir.
    (David Allsopp, review by Nicolás Ojeda Bär, Sébastien Hinderer and
    Xavier Leroy)

  • [breaking change] #9765: add init functions to Bigarray.
    (Jeremy Yallop, review by Gabriel Scherer, Nicolás Ojeda Bär, and
    Xavier Leroy)

  • [breaking change] #9668: List.equal, List.compare
    (This could break code using “open List” by shadowing
    Stdlib.{equal,compare}.)
    (Gabriel Scherer, review by Nicolás Ojeda Bär, Daniel Bünzli and Alain Frisch)

  • #9066: a new Either module with
    type 'a Either.t = Left of 'a | Right of 'b
    (Gabriel Scherer, review by Daniel Bünzli, Thomas Refis, Jeremy Yallop)

  • #9066: List.partition_map :
    ('a → ('b, 'c) Either.t) → 'a list → 'b list * 'c list
    (Gabriel Scherer, review by Jeremy Yallop)

  • #9865: add Format.pp_print_seq
    (Raphaël Proust, review by Nicolás Ojeda Bär)

Compiler user-interface and warnings (highlights):

  • #9657: Warnings can now be referred to by their mnemonic name. The names are
    displayed using -warn-help and can be utilized anywhere where a warning list
    specification is expected.
    ocamlc -w +fragile-match
    …[@@ocaml.warning “-fragile-match”]
    Note that only a single warning name at a time is supported for now:
    “-w +foo-bar” does not work, you must use “-w +foo -w -bar”.
    (Nicolás Ojeda Bär, review by Gabriel Scherer, Florian Angeletti and
    Leo White)

  • #8939: Command-line option to save Linear IR before emit.
    (Greta Yorsh, review by Mark Shinwell, Sébastien Hinderer and Frédéric Bour)

  • #9003: Start compilation from Emit when the input file is in Linear IR format.
    (Greta Yorsh, review by Jérémie Dimino, Gabriel Scherer and Frédéric Bour)

Language features (highlights):

  • [breaking change] #9500, #9727, #9866, #9870, #9873: Injectivity annotations
    One can now mark type parameters as injective, which is useful for
    abstract types:
    module Vec : sig type !'a t end = struct type 'a t = 'a array end
    On non-abstract types, this can be used to check the injectivity of
    parameters. Since all parameters of record and sum types are by definition
    injective, this only makes sense for type abbreviations:
    type !'a t = 'a list
    Note that this change required making the regularity check stricter.
    (Jacques Garrigue, review by Jeremy Yallop and Leo White)

Runtime system (highlights):

  • #9534, #9947: Introduce a naked pointers checker mode to the runtime
    (configure option --enable-naked-pointers-checker). Alarms are printed
    when the garbage collector finds out-of-heap pointers that could
    cause a crash in no-naked-pointers mode.
    (Enguerrand Decorne, KC Sivaramakrishnan, Xavier Leroy, Stephen Dolan,
    David Allsopp, Nicolás Ojeda Bär review by Xavier Leroy, Nicolás Ojeda Bär)

  • [breaking change] #1128, #7503, #9036, #9722, #10069: EINTR-based signal handling.
    When a signal arrives, avoid running its OCaml handler in the middle
    of a blocking section. Instead, allow control to return quickly to
    a polling point where the signal handler can safely run, ensuring that
    I/O locks are not held while it runs. A polling point was removed from
    caml_leave_blocking_section, and one added to caml_raise.
    (Stephen Dolan, review by Goswin von Brederlow, Xavier Leroy, Damien
    Doligez, Anil Madhavapeddy, Guillaume Munch-Maccagnoni and Jacques-
    Henri Jourdan)

  • [breaking change] #5154, #9569, #9734: Add Val_none, Some_val, Is_none, Is_some,
    caml_alloc_some, and Tag_some. As these macros are sometimes defined by
    authors of C bindings, this change may cause warnings/errors in case of
    redefinition.
    (Nicolás Ojeda Bär, review by Stephen Dolan, Gabriel Scherer, Mark Shinwell,
    and Xavier Leroy)

  • [breaking change] #9674: Memprof: guarantee that an allocation callback is always run
    in the same thread the allocation takes place
    (Jacques-Henri Jourdan, review by Stephen Dolan)

  • #10025: Track custom blocks (e.g. Bigarray) with Memprof
    (Stephen Dolan, review by Leo White, Gabriel Scherer and Jacques-Henri
    Jourdan)

  • #9619: Change representation of function closures so that code pointers
    can be easily distinguished from environment variables
    (Xavier Leroy, review by Mark Shinwell and Damien Doligez)

  • #9654: More efficient management of code fragments.
    (Xavier Leroy, review by Jacques-Henri Jourdan, Damien Doligez, and
    Stephen Dolan)

Other libraries (highlights):

  • #9573: reimplement Unix.create_process and related functions without
    Unix.fork, for better efficiency and compatibility with threads.
    (Xavier Leroy, review by Gabriel Scherer and Anil Madhavapeddy)

  • #9575: Add Unix.is_inet6_addr
    (Nicolás Ojeda Bär, review by Xavier Leroy)

  • #9930: new module Semaphore in the thread library, implementing
    counting semaphores and binary semaphores
    (Xavier Leroy, review by Daniel Bünzli and Damien Doligez,
    additional suggestions by Stephen Dolan and Craig Ferguson)

  • [breaking change] #9206, #9419: update documentation of the threads library;
    deprecate Thread.kill, Thread.wait_read, Thread.wait_write,
    and the whole ThreadUnix module.
    (Xavier Leroy, review by Florian Angeletti, Guillaume Munch-Maccagnoni,
    and Gabriel Scherer)

Manual and documentation (highlights):

  • #9755: Manual: post-processing the html generated by ocamldoc and
    hevea. Improvements on design and navigation, including a mobile
    version, and a quick-search functionality for the API.
    (San Vũ Ngọc, review by David Allsopp and Florian Angeletti)

  • #9468: HACKING.adoc: using dune to get merlin support
    (Thomas Refis, review by Gabriel Scherer)

  • #9684: document in address_class.h the runtime value model in
    naked-pointers and no-naked-pointers mode
    (Xavier Leroy and Gabriel Scherer)

Internal/compiler-libs changes (highlights):

  • #9464, #9493, #9520, #9563, #9599, #9608, #9647: refactor
    the pattern-matching compiler
    (Thomas Refis and Gabriel Scherer, review by Florian Angeletti)

  • #9696: ocamltest now shows its log when a test fails. In addition, the log
    contains the output of executed programs.
    (Nicolás Ojeda Bär, review by David Allsopp, Sébastien Hinderer and Gabriel
    Scherer)

Build system (highlights):

  • #9824, #9837: Honour the CFLAGS and CPPFLAGS variables.
    (Sébastien Hinderer, review by David Allsopp)

  • #10063: (Re-)enable building on illumos (SmartOS, OmniOS, …) and
    Oracle Solaris; x86_64/GCC and 64-bit SPARC/Sun PRO C compilers.
    (partially revert #2024).
    (Tõivo Leedjärv and Konstantin Romanov,
    review by Gabriel Scherer, Sébastien Hinderer and Xavier Leroy)

The most important change, if you ask me, is from @nojb: support for mnemonic names for warnings instead of numbers. [@warning "-fragile-match"] is just so much better than [@warning "-4"]. Now we can locally disable/enable a warning.

6 Likes
3

Quick question: do we have a release candidate for documentation too? I want to get more information about so-called ‘Linear IR’

4

Linear-IR is a low-level intermediate representation of the compiler, which you can see for a given module by passing the -dlinear option to ocamlopt. It is lower-level than Cmm -dcmm which we typically inspect for performance reasoning.

It is mentioned in the Changes because Greta Yorsh (@gretay-js), from Jane Street, is experimenting with code-layout optimizations for OCaml binaries, implemented as an external tool that transforms this linear-IR representation of programs. To read more about it (keyword is “Feedback-Directed Optimization (FDO) for OCaml”, see the ocamlfdo repository; online I also found this podcast episode and slides from a talk.

I don’t think that this features is currently documented in the OCaml manual, which is unfortunate. (The github issues themselves are not great to learn more about it, because the interface went through several iterations and their first post is out of date.) I will ask whether this can be improved (edit: done).

1 Like
5

Well, OCaml-4.11.0 is still the champion:

  • alpha1 - alpha2 - alpha3
  • beta1 - beta2 - beta3
  • rc1 - rc2

Quite an accomplishment…

6

The first release candidate for 4.11.0 has never been published (there was a bug discovered while the publication process was on-going). But yes, 4.11.0 is still the champion with: 3 alpha, 3 beta, and 1 rc.

Before we had:

  • 4.10.0: 2 betas, 2 rcs
  • 4.09.0: 2 betas, 0 rc
  • 4.08.0: 3 betas, 2 rcs
  • 4.07.0: 2 betas, 1 rc
  • 4.06.0: 2 betas, 1rc

The main change is thus the new alpha release phase.
This new phase corresponds to a previously private testing phase after a version was deemed feature complete. My experience working with @kit-ty-kate on 4.10.0 is that this private phase created a certain amount of friction between the ideal calendar for the compiler and the one for the ecosystem. In particular, it was not clear at which point of the release cycle, core tools should have been updated to work with the new version. Before the beta, there was always a perceived risk that compiler internals would change under the feet of dune or merlin. Conversely, if core tools were only updated after the beta, this means that for a number of weeks, the beta was unusable for anyone.

The new alpha releases are here to make this private testing phase public, and give to core tools both a synchronization point for updates, and a windows for protesting if some core changes make their life impossible.

Between this new alpha release phase and the hard work of @kit-ty-kate and the rest of the ocaml readiness team, the two last release cycles has been going quite smoothly. And I think for 4.12.0, we are at a point when people can switch to 4.12.0 on the release day.

4 Likes
7

In my opinion you forgot the following in the highlights:

  • [breaking change] #9757, #9846, #10161: check proper ownership when operating over mutexes. Now, unlocking a mutex held by another thread or not locked at all reliably raises a Sys_error exception. Before, it was undefined behavior, but the documentation did not say so. Likewise, locking a mutex already locked by the current thread reliably raises a Sys_error exception. Before, it could deadlock or succeed (and do recursive locking), depending on the OS.
    (Xavier Leroy, report by Guillaume Munch-Maccagnoni, review by Guillaume Munch-Maccagnoni, David Allsopp, and Stephen Dolan)

I mention it given the breaking character of the change. It is already known to affect a high-profile library and it is suspected to affect another, and we only know this thanks to attention being indirectly brought to it by chance on this forum a while ago.

You might see an undocumented corner-case, but the documentation was not explicit and writing software for the real world has the annoying tendency to create corner cases. More generally, programmers are already expected to somehow evaluate the notes of every minor release for breaking changes that might affect them, and in my opinion hiding breaking changes in the middle of boring innocuous ones kind of defeats the purpose of these highlights.

8

Is that really a breaking change? I would worry if a particular project thought that mutexes didn’t have ownership semantics and could be treated as binary semaphores.

9

At least one renowned project voluntarily used it as a semaphore, and it was probably accidentally well-defined behaviour on common platforms due to the additional protection granted by the global lock. Another was a well-defined recursive locking behaviour on Windows. (This was counter-intuitive to me, and not all changes marked as breaking in the release notes are as clearly breaking.)

On the change itself, let me add that this is a very desirable clarification and improvement, which might avoid hard-to-find bugs in the future. Furthermore, when learning that Mutex was being used as a semaphore in production code, several solutions were envisioned, and the solution to introduce semaphores separately was retained. Thanks to this, there is a new Semaphore module in the standard library. (The affected project has already converted to use the new module.)

10

[…]

Why do you shred so much mystery on the names of the affected projects ? Are they private ? I’m sure most people around here would be interested in how the bug and fix look.

11

Irmin is one project that was affected by this change (and, by extension, Tezos), as we were indeed using the standard library mutexes as if they were binary semaphores in several places. This happened fairly naturally as the code in question expanded scope, i.e.

  • initially we were using the mutex just for a certain critical section in the code;
  • later we decided that the process might (optionally) want to delegate to a child thread to finish the expensive tail-end of this critical section;
  • this left us with a mutex that was taken by the main thread, which did some initial work in the critical section and launched a worker thread to finish the job and release the mutex.

This state worked fine for several releases, at least when relying on glibc. Although I didn’t know it at the time, on other platforms (e.g. FreeBSD) this would result in a raised exception. The 4.12 change to error-checking mutexes caused a similar hard failure on all platforms.

Our response to this issue was to extract the semaphore-compat library from @xavierleroy’s implementation of the Semaphore module in OCaml 4.12 and use that in our datastructure implementation instead.

5 Likes
12

Looks like opam isn’t compatible yet? (This PR #4437 wasn’t merged).

13

Seems fine as #4434 was merged, and is a superset of the one you mention.

1 Like