[ANN] ppx_seq v0.1.1

Hello everyone, my first contribution to opam-repository has just been merged and is waiting to hit the caches of opam.ocaml.org.

ppx_seq is a cute un-intrusive literal syntax for Seq. The rewriter is simple and has very small surface area: just [%seq x; y; z; ...] and [%seq.empty].
It tries to be maximally compatible with all OCaml releases from 4.07 (when Seq was introduced) to 4.14 and beyond

The reason I created this rewriter is to make it an easier choice to reach first for Seq as a general data structure (instead of e.g. list). That wasn’t quite attractive before because of how minimal the Seq module was, it was mostly used as an intermediate step between two types of collections, but now with 4.14 about to be released, Seq is becoming a first-class data structure with a very versatile API.

I hope my little rewriter helps make it even more attractive to use. Check it out and maybe leave me some feedback. Thanks <3

15 Likes

That’s a nice idea! A few remarks:

  • you should commit the opam file, so people can opam pin your repository.
  • markdown is nice for readme files :slight_smile:
  • you can be compatible with pre-4.07 versions of OCaml, by simple requiring the generate code to depend on the compatibility package seq.
  • it’d be nice (but maybe harder) to implement basic comprehensions for sequences. I’m not sure what a OCaml-compatible syntax would be, though.
2 Likes

The documentation should make it clearer what the semantics is. You have chosen to translate [%seq a; b] to

fun () -> Seq.cons (a, fun () -> Seq.cons (b, fun () -> Seq.nil))

but some users might have expected it to be translated to

let a = a in let b = b in
fun () -> Seq.cons (a, fun () -> Seq.cons (b, fun () -> Seq.nil))

This matters a lot for side effects.

5 Likes

If I understand you correctly, your suggestion is that I should better clarify that side-effectful code gets properly delayed, right?
That way people don’t expect side-effectful code to fire before being Seq’d like here in that first line…

(i.e. for this test to fail)?

The rewritten output is indeed fun () -> Seq.Cons (a, ...). The wrapping under fun () -> ... would prevent any side-effects from firing and I imagine that this is the intended behaviour of Seq. I might’ve assumed too much by thinking being implicit about that wouldn’t cause uncertainty. Thanks!

1 Like

Compatibility has increased to OCaml 4.04.2, now only bounded by ppxlib itself.
However, I made seq a test-only dependency to give people more freedom to override with their own custom Seq module without having to have seq around in the dependency list… Although I admit that’d be such a niche setup.

Still not sure about whether allowing such overriding is a good or bad thing too. Only time and tickets will tell haha

as for comprehensions, I believe that is covered with the more general letop syntax, and I’d rather keep it like that instead of rolling my own. It doesn’t sound too hard (I have some thoughts… maybe I can (ab)use the record syntax to get a nice comprehensions dsl…) but it feels like it isn’t within the domain of the rewriter… what do you think?

Something I’m more interested in is a range syntax though. e.g. [%seq a..b] and [%seq a, b..c]

2 Likes

Also! there appears to be a package called ppx_monadic that does exactly those comprehensions. It’s not maintained anymore though (at least it seems so).

It would be interesting to extend this PPX to support pattern matching on Seq.t values, in the style of the old Stream parsers, see parsers.

Cheers,
Nicolas

4 Likes

Yes. But saying that the side effects are delayed is not sufficient. There are basically three different semantics, and they all make sense. [%seq a; b; ...] could be any of the following:

  1. List.to_seq [a; b; ...], that is, side effects are performed eagerly and only once.
  2. Seq.map Lazy.force (List.to_seq [lazy a; lazy b; ...]), that is, side effects are performed on demand, but only once.
  3. Seq.map (fun f -> f ()) (List.to_seq [fun () -> a; fun () -> b; ...]), that is, side effects are performed on demand, but repeatedly.
2 Likes

Thanks for pointing that out, I made that point that it’s just plain Seq without any memoization or eager evaluation clear.

Excellent suggestion, I’ve been thinking about that yesterday actually!

@nojb , are you suggesting that it might be useful to port the stream-parsers to work on Seq.t ? B/c that’s probably not very difficult to do. Indeed, it would suffice to just make a wrapper module (call it “SeqStream”) that had the same API as what the stream-parsers use – which is pretty minimal.

Well, that would be an orthogonal project. What I was suggesting is to have a lightweight ppx syntax for seq/stream parsers which does not depend on camlp{4,5}. That syntax was convenient for writing recursive descent parsers, but depending on camlp{4,5} for this is overkill.

Cheers,
Nicolas

1 Like

Wouldn’t it be better to have the pattern matching over Seq built into OCaml? It will be almost a complete replacement to stream parsers.

Especially, since making ocamllex to return a Seq.t is quite easy.

Perhaps, but IMO this would come later. First this needs to be experimented with outside of the compiler; once that is done and a consensual design (and implementation!) is ready, we could consider the question of upstreaming…

Cheers,
Nicolas

fwiw, pyret has an interesting approach to generic collection literals, which I plan to replicate once the experiment with ppx_seq proves frictionless. their approach allows you to construct all kinds of collections using a uniform syntax.

If the generic collections ppx turns out popular, maybe it’d be good to include it eventually as some lightweight sugar for OCaml… perhaps something like Seq.[1;2;3], LazyList.[1;2;3], IntSet.[1;2;3], IntMap.[1, "one"; 2, "two"] etc… where intermediate lists aren’t constructed, and delay semantics (or any kind of pre-construction operations really) are respected.

1 Like

Obviously questions of commodious syntax are ripe for de gustibus non est disputandem. But I think you’ll find it difficult to come up with a nice syntax for “seq parsers” while staying within the syntax of OCaml. While I agree that for many, many uses one doesn’t need to full power of Camlp5 (PPX attributes & extensions suffice), there are others where it’s simply necessary, and stream-parsers is one.

But it’s all good, I’m not trying to convince anybody.

1 Like

By the way, @silene, would you mind taking a look at the newest release docs? Documentation · ppx_seq 0.3.1 · OCaml Packages
I tried to clarify as much as possible the semantics of the macros. Any feedback is appreciated!

No ambiguity, this looks good.

1 Like