[ANN] mirage-crypto 0.11.3 (with more speed for elliptic curves) -- and the future roadmap of mirage-crypto

Ecosystem
,
1

Hey,

we’re happy to announce mirage-crypto 0.11.3 (which just got merged to opam-repository), which includes huge performance improvements for elliptic curves. The API didn’t change at all :slight_smile:

Background

Mirage-crypto, despite its name, is a crypographic library for OCaml. It provides hash functions (MD5, SHA1, SHA2), symmetric ciphers (DES, AES, ChaCha20, RC4), asymmetric cryptography (RSA, DSA, DH, ECDSA/EdDSA, ECDH). It originates from nocrypto (see the blog post from 2014) by David Kaloper.

You can use mirage-crypto in any OCaml application on Linux, macOS, or Windows.

release 0.11.3

The background story is that we finally merged the “use bytes instead of Cstruct.t” PR which was opened ~2.5 years ago by @dinosaure. We reviewed that, and did some benchmarks. And even went a bit further and are now using string (instead of bytes). See https://blog.robur.coop/articles/speeding-ec-string.html for further details.

Another PR worth mentioning is use windowed algorithm for base scalar multiplication from Virgile Robles – now some precomputed tables are shipped (same approach was done for 25519 already).

See this PR (and the release notes) for some detailed performance numbers on different CPUs – the P256 sign operation is around 10x faster than older releases. This is still 5 times slower than OpenSSL - but then we use fiat-crypto instead of handcrafted assembly code. We’re keen to improve the performance even further – ideas, observations, experiments and PRs are very welcome. We investigated benchmarking of e.g. digest algorithms across the OCaml ecosystem and OpenSSL as baseline and welcome improvements and further work on that (especially AES-GCM and Poly1305-ChaCha20 are painfully slow compared to OpenSSL).

Other improvements and fixes include support for Loongarch, NetBSD, use rdtime instead of rdcycle on RISC-V when in user mode, initial support for CL.EXE. Thanks to everyone involved in this released: @jbeckford @reynir @dinosaure @palainp @edwin

The full changelog may be worth to read.

Future roadmap (breaking changes)

Also, please note if you’re using mirage-crypto that we’ll revise the API and no longer use Cstruct.t / bigarrays, but instead bytes/string. 0.11.3 will be the last release using Cstruct.t. The hash functionality (Mirage_crypto.Hash) will also be removed (since digestif implements them nicely). Please voice your concerns / ideas at Roadmap for 1.0 release · Issue #205 · mirage/mirage-crypto · GitHub

10 Likes
2

note: updated above text for some context what this package is and its history

3

I find this fascinating. If the mirage ecosystem moves away from Cstruct.t and towards string/bytes, I’ll personally be happier using the mirage libraries.

A downside of bytes, though, is that there’s no existing slice type in common use (by “slice” I mean a sub-range of bytes that cheap to produce and shares the actual byte array with other slices, just like Cstruct.t does for bigarrays. Would there be a space for a kind of Bstruct.t or Byte_slice.t or similar?

4

Would there be a space for a kind of Bstruct.t or Byte_slice.t or similar?

Actually astring has a slice type String.Sub.t with plenty of useful functions that cstruct has been copied to offer a better user experience.

5

Great question, I was pointed to [Experiment] Cstruct to Bstruct by patricoferris · Pull Request #101 · ocaml-multicore/ocaml-uring · GitHub We’ll see how the mirage-crypto moving goes, and whether we need a sub-range of bytes. Another path is to pass an offset through. My gut feeling is that the whole bounds check business in cstruct (ocaml-cstruct/lib/cstruct.ml at v6.2.0 · mirage/ocaml-cstruct · GitHub – as well as at the underlying layer (bigarray / caml_ba_ref_1) is hindering getting nice performance.

For the “the MirageOS ecosystem moves away from Cstruct.t / bigarray”, please let me rephrase from the blog post https://blog.robur.coop/articles/speeding-ec-string.html – OCaml stdlib got more enhanced in terms of string/bytes and retrieving parts as big/little endian signed/unsigned integers. Also, I like the API of digestif, where functions are defined on string, bytes, and bigarray. That may (for some packages) be a suitable solution.

6

Just to add that the slight peculiarity with the Bstruct experiment linked was that it was in the context of using Uring. This required the pointer to the bytes to be non-moving which is something you get for free with Cstruct using bigarrays which are allocated in the C heap. Byte arrays (i.e. bytes) of a certain size are currently non-moving in OCaml 5+, but that is relying on an implementation detail of the OCaml runtime as far as I understand it.

Maybe this doesn’t matter too much at all and doesn’t come up that often. Just wanted to add that information in case someone is looking at it.

7

It is nice when we can zero out memory after it is used, especially in a security library when keys (etc.) are in RAM. I believe Apple did this by default for the C free() function a year ago: iOS 16.1 Release Notes - Memory Allocation.

With GC-moveable memory values like string most of the zeroing operations become ineffective. Perhaps adding the string as a GC root might make it immovable; regardless, it would be good to document a warning for security keys so people are aware. (If I’m understanding the implications of the switch away from cstruct correctly; someone correct me if I’m wrong)

8

That would definitively be great! We should convince ourselves (by reading code and writing tests) that we’re able to “zero out” memory where secrets are stored. As a note, even with Cstruct.t / bigarray we’re not zeroing out anything as of today (and some RSA/DSA/DH computations use zarith which doesn’t zero out or tracks the allocated limbs [as far as I know]).

Unfortunately, it is also not easy: on the one hand you need to pass these secret keys as input at some point (and there need to be sure to clear out the memory), on the other hand sometimes you want to use the secrets (esp. for asymmetric crypto) and need to ensure that all derived numbers (pieces of memory) are properly cleared.

This is an important topic which should be rigorously addressed. I’m sure there’s prior work in forms of some thesis or papers around. I haven’t seen any work in respect to OCaml unfortunately (if someone has, would be great to point to it). My gut feeling is that “using string instead of Cstruct.t” doesn’t make it worse, but we may as well keep bigarrays for the secrets. I added a comment about it to the 1.0 roadmap issue.

1 Like