[ANN] Sedlex 3.8.1

Community
, , ,
1

Sedlex 3.8.1 is out! :tada:

Sedlex is a unicode-friendly lexer generator for OCaml originally written by
Alain Frisch at LexiFi and now maintained under the
ocaml-community umbrella.

The headline feature is named capture groups. Sub-patterns can be bound
with OCaml’s as syntax and read back in the action:

match%sedlex buf with
| (Plus ('0'..'9') as num), '.', (Plus ('0'..'9') as frac) ->
    let n = Sedlexing.Utf8.of_submatch num in
    let f = Sedlexing.Utf8.of_submatch frac in
    Printf.printf "integer=%s fractional=%s\n" n f
| _ -> ()

Captures are implemented with a tagged DFA, so the generated lexer remains a
single pass with no backtracking, and lexers that don’t use as generate the
same code as before. Sub-matches are leftmost-greedy within the longest match.
as is not allowed under Star, Plus, Opt, Rep, Compl, Sub or
Intersect.

This release also brings, among other things:

  • Unicode 18.0.0
  • Faster lexeme, sub_lexeme and of_submatch for Latin1, UTF-8 and UTF-16

You can read the notes in
Release 3.8 · ocaml-community/sedlex · GitHub (unreleased) and
Release 3.8.1 · ocaml-community/sedlex · GitHub.

It should be available soon on opam.

7 Likes