V0.9 release of Jane Street packages

Dear OCaml developers,

I’m happy to announce the v0.9 release of Jane Street packages! This
release comes 13 months after the last stable one and is packed with a
ton of new stuff.

We are now releasing 94 packages, so 32 more than the previous
release. New packages cover a wide range of areas, such as shell
programming, web programming, standard libraries, Emacs hacking and
more.

Given its size and the increasing number of users, getting this
release out in opam was a challenge. Especially I thank Anil
Madhavapeddy for his help with the testing and fixing of reverse
dependencies.

All the packages are now in opam and the API documentation is
available on our website:

https://ocaml.janestreet.com/ocaml-core/v0.9/doc/

We don’t have public release notes for this release yet. However, we
are in the process of reviewing the past year of change logs and hope
to publish them online soon. In the meantime, here is a summary of
major changes and new features available in this release.

Simpler versioning

The first noteworthy change is the versioning scheme. We used to
assign version numbers based on our internal ones. However these
didn’t make much sense outside of Jane Street, and especially there
was a huge gap between each releases. So we are now using a more
classic versioning scheme.

In order to keep version numbers going up, they had to be prefixed
with a ‘v’. And because this release introduce a new standard library,
Base, with an API that is still under active development, the new
versions start with v0.9.0.

Faster and more portable builds

This release is the first one to use Jbuilder [1], a new build system that
was initially designed to ease the publication of our packages. The
main consequences for users are:

  • much faster compilation times. It has been observed that Jane Street
    packages such as Core are now 6 times faster to build and install

  • improved portability. For instance packages whose code is portable
    build on Windows with nothing more than a working OCaml compiler

Regarding portability, this release introduces configurator [2], a small
but convenient package that helps finding out available system
features. We started using it systematically in packages where we have
C stubs.

[1] https://github.com/janestreet/jbuilder
[2] https://github.com/janestreet/configurator

Better compatibility with multiple versions of OCaml

Since the switch to ppx our packages used to be stuck with one version
of the compiler. This was due to our heavy use of ppx rewriters and to
the fact that each version of OCaml tends to break code using the
compiler libraries.

In this release all our ppx code is based on the
ocaml-migrate-parsetree [1] library. As a result our packages now
build with OCaml from 4.03 to 4.06.

There are still some issues related to ppx versioning that will need
one more round of refactoring to be solved.

[1] https://github.com/let-def/ocaml-migrate-parsetree

New lighter, portable and guilt free standard library

This new release introduces Base [1], a wholesale replacement of the
standard library distributed with OCaml. It aims to provide better
standards and conventions, while only offering fully portable
features.

Base is still under active development, and work on the API is not yet
finished. However, it was initially developed mostly by reorganising
code from Core_kernel and it is the basis for all the Jane Street code
base, meaning that it is carefully reviewed and heavily tested. Using it
in old and new code is a breeze.

Note that to be fully portable Base doesn’t expose IO
operations. These are provided by the companion Stdio library [2].

[1] https://github.com/janestreet/base
[2] https://github.com/janestreet/stdio

And more…

Following is a brief overview of the other new packages available in
this release, with more details available on their respective home
pages.

  • Ppx_hash

Automatic generation of Hash functions from type expressions and type
definitions.

https://github.com/janestreet/ppx_hash

  • Bin_prot shapes

An extension to bin_prot to check safe use of deserialization.

https://github.com/janestreet/bin_prot/tree/master/shape

  • Incr_dom

A library for building dynamic webapps, using Js_of_ocaml.

https://github.com/janestreet/incr_dom

  • Incr_map

Helpers for incremental operations on map like data structures.

https://github.com/janestreet/incr_dom

  • Incr_select

Handling of large set of incremental outputs from a single input

https://github.com/janestreet/incr_select

  • Virtual_dom

OCaml bindings for the virtual-dom library

https://github.com/janestreet/virtual_dom

  • Shexp

Shexp was initially intended as a S-expression based shell to replace
bash in Makefile based build for Jane Street packages. However, this
project was superseded by Jbuilder. What’s left is a nice process
monad allowing one to construct complex and typed process pipelines.
Shexp is still in its infancy but has already been successfully used
for various purposes.

https://github.com/janestreet/shexp

  • Spawn

Essentially an improved version of Unix.create_process, implemented
using vfork on Unix, which is much more efficient than fork.

https://github.com/janestreet/spawn

  • Posixat

Bindings to the various *at posix functions.

https://github.com/janestreet/posixat

  • Ecaml

OCaml plugins for Emacs.

https://github.com/janestreet/ecaml

  • Expect_test_helpers_kernel

Various helpers for writing expectation tests.

https://github.com/janestreet/expect_test_helpers_kernel

  • Parsexp

Lighter S-expression parsing library, with a more consistent API and
better behaved in JavaScript.

https://github.com/janestreet/parsexp

  • Cinaps

Trivial meta-programming a la expect-test.

More here: https://blogs.janestreet.com/trivial-meta-programming-with-cinaps/

https://github.com/janestreet/cinaps

  • Ppx_optional

Match statements for zero allocation options.

https://github.com/janestreet/ppx_optional

  • Sexp_pretty

A S-expression prettifying library.

https://github.com/janestreet/sexp_pretty

  • Topological_sort

Single-module library that implements a simple topological-sort algorithm.

https://github.com/janestreet/topological_sort

9 Likes

Could you elaborate on the relationship between the new Base and the existing Core libraries? Is Base meant to replace Core in the near future? If so, does that mean that we can expect Core to be deprecated initially, and will have to port existing code using Core to Base?

Briefly:

  • Base: minimal stdlib replacement. Portable and lightweight and intended to be highly stable.
  • Core_kernel: Extension of Base. More full featured, with more code and dependencies, and APIs that evolve more quickly. Portable, and works on Javascript.
  • Core: Core_kernel extended with UNIX APIs.

None of these are going away. The most disruptive thing that we’ve discussed is adjusting the boundaries between Core_kernel and Core, e.g., Core may become the portable one, with unix dependencies moved out to a separate package. That said, such a shuffle is not coming soon.

3 Likes

Ahh thanks for clearing that up!

I upgraded my local install to v0.9.1 of the Core libraries, but when trying to recompile a project against it, I get the error: Cannot locate deriver sexp. The configure script for that project (generated via oasis, I think) seems to locate px_tools and ppx_sexp_conv just fine.

Try installing ppx_deriving. You need it in order to use ppx_type_conv via ocamlfind ocamlc -package ppx_sexp_conv .... Maybe we should add a post install message…

I think that might be useful; it’s a common FAQ with the new release.