Application-specific Improvements to the Ecosystem

re: python FFI, I seem to remember that at least @thierry-martinez has done quite a bit of work on this. A couple links I could dig up from my browser history (there might be more, I haven’t used the stuff myself): GitHub - thierry-martinez/pyml: OCaml bindings for Python ,
GitHub - thierry-martinez/ocaml-in-python: Effortless Python bindings for OCaml modules (based on pyml),
GitHub - mooreryan/ocaml_python_bindgen: Generate Python bindings via pyml from OCaml value specifications (also based on pyml).

1 Like

Thanks for that. For the next relevant project that comes up, I’ll check it out and see how seemless bindgen has made things and how reliable it is compared to what’s available in other ecosystems.

Sounds good. I haven’t had yet the need for python interop myself, but I’d be interested to read an experience report if you do try.

I’ll write up that report when I do get a project that’s a good fit.

Browsing through the docs, I wish it was a bit more comprehensive and automated since you still have to supply types for the val specs (contrast with Julia’s PyCall), but we gotta start somewhere and I’ll see how much friction there is in practice and how much difference there is between the two once I give it a try with something substantial.

We think about “regexp support” as just “bindings”. Or maybe “bindings”+“match construct”. But look at the extensive support in Perl for regular expressions – all the various flags that are available and what they do. And the concise yet transparent way in which these things are invoked. OCaml has nothing to compare. In that sense, yes, the pcre bindings aren’t that great.

But I don’t to slag on them as bindings: they’re perfectly fine as bindings. What’s missing is language support to make using those bindings as effortless as they are in Perl.

And something else that I must note: that language support is going to necessarily mean stepping outside the current syntax of OCaml. I mean, it’s simply not going to be possible (I think) to provide the same level of effortlessness as you get in Perl, within the current OCaml syntax (hence, via hacking that syntax with PPX). I could be wrong about this. But for instance, sedlex’s PPX syntax incarnation is decidedly less effortless than ocamllex.

I agree. I use OCaml in a scientific context for simulations – but then I need to export results to CSV or some such and read that from python to do proper visualization, as that is not available in OCaml (No, the plotting facilities currently in Owl are not proper visualization. I need functionality on par with matplotlib). This workflow feels clunky compared to a Jupyter notebook, and really is somewhat painful.

Now if I could make a wrapper to drive my OCaml simulation from python with almost no effort and automatic conversion of arrays/bigarrays to numpy arrays, that pain would go away. I’ve been wanting to try pyml and friends but it did still appear to require some non-negligible wrapping work.

In an ideal world, most of the numerical / data science ecosystem around python would have bindings for OCaml, but that’s not going to happen – the next best thing really is painless python interoperability.

I disagree with the notion put forward elsewhere that static typing makes OCaml unsuitable for exploratory numerical/data science work. Once a scientific program evolves over more than a week or so, being forced to structure the code sensibly by the type system already pays off IME.

5 Likes

it’s … unfair? It is what it is. Whose feelings are we trying to spare by caring about fairness? Perl didn’t stumble into being good for oneliners, it saw them in awk and imitated the required features. Ruby did the same, and has the same flags as Perl. The D programming language was trying to be a better C++, and not specifically designed for one-liners, but it comes with ‘rdmd’, a little wrapper for the D compiler that can be used as such:

$ rdmd --loop='line.replaceFirst("\r$", "").writeln'
hi^M
hi
there^M
there

where the lines ctrl-M endings are my input. This isn’t exactly the same usage as Perl (which is also using the <ARGV> feature of defaulting to stdin input, but given arguments taking each of them as filenames to use for input) but it’s convenient enough that you might write it instead of Perl.

With the same caveat:

#! /usr/bin/env ocamlscript
Ocaml.packs := ["str"]
--
let rec pie file f =
  match In_channel.input_line file with
  | Some line ->
    print_endline (f line);
    pie file f
  | None -> ()

let () =
  let dosline = Str.regexp "\r$" in
  pie stdin (Str.replace_first dosline "")

This is slightly annoying as ocamlscript requires camlp4 requires ocaml <= 4.14.0, and my In_channels use requires 4.14.0, so you practically need a whole opam switch just for this one script. Buliding with dune saves you from the camlp4 requirement but now you have a bunch of files instead of Perl’s zero files for the task. What I’d suggest is either extending or competing with ocamlscript and adding flags for some usage like

rocaml --let=dosline='Str.regexp "\r$" --loop='Str.replace_first dosline ""'

No design changes needed to the language. Even without such a tool, sysadmins don’t sleepwalk into creating git repos and starting to make more polished, safer, more maintainable scripts. At least the process of creating the git repo usually wakes them up. So they’re no longer as interested in oneliners but are still interested in the functionality, and here even the full dune implementation of dos2unix might be compelling enough: GitHub - jrfondren/dos2unix-ocaml: An example dos2unix in OCaml

1 Like

I have used the matplotlib bindings for a simple project and they were quite usable. They where also very incomplete, but it is very easy to add the missing stuff, it took me like 20 min to add the function I wanted, and this include the time needed to understand the way the codebase worked in the first place.

3 Likes

I just tried these, and they look promising. Is there a way to have a window open with graphics being displayed along a utop session? So far I was able to do savefig but no interactive work.

Update: Mpl.show () does show the plot in a window, but that has to be closed again for the utop session to continue.

1 Like

To emphasize what has been said earlier, I find the OCaml/Python bindings found in ppx_python and pyml to be extremely convenient. You can seamlessly derive python bindings from type definition both ways. I would be curious to see which kinds of improvements you’d like to have over this.

3 Likes

Indeed ppx_python looks pretty sweet. When I did some cursory testing around py.ml, I got the following setup working:

  • Define an OCaml module containing some computational code
  • Using py.ml define a Python module that contains python wrapper functions around the OCaml code
  • Start a Python interpreter with Py.run.interactive and use the wrappers.
  • Once the interpreter is started, there is no going back to the OCaml side.

What I would have liked in addition would be to have a way to have two interactive loops concurrently: One (utop) where I can inject new OCaml-side functions and new python wrapper definitions, and another (ipython) that calls the (current versions of the) wrappers. This would allow a nice iterative development cycle.
Maybe there is a reason why this cannot work? Or I missed something obvious? In any case I wasn’t able to achieve it.

1 Like

I have never heard of ocamlscript. Is this something that would be useful to bring up-to-date? I’m sure updating it to use camlp5 should be straightforward, and I’d be happy to do that, if it would be useful.

1 Like

I put forward the notion, and I left out the other part of it, which is that I agree with your statement here. Once code grows, it benefits from a strong type system. The core of data science, however, cannot benefit much from OCaml’s type system (unfortunately). There are also some design decisions in Owl which I think hurt the library’s usability somewhat in terms of data science.

1 Like

I never got to use it so I cannot judge, but I’m interested in that comment. Could you please elaborate ?

5 Likes

Personally, I often use the monadic Result type together with a polymorphic variant for the actual errors. This makes dealing with errors from different “levels” of my software (library, command-line tool, and GUI) quite comfortable (and type-safe!).
@keleshev has written a nice blog post on this: Composable Error Handling in OCaml with a recent follow up: Advanced Error Handling in OCaml

10 Likes

This is a great piece for beginners like myself. I’ve only ever done the variant error method, and it certainly isn’t as pretty as using polymorphic variants.

These are the type of writing I’d appreciate seeing more often surfaced on ocamlverse, the awesome-ocaml GitHub, or ocaml blog. I get that there’s a wide spectrum of ocaml users (apparently skewed more on experienced based on the active users here), but maybe a tag to categorize skill level can work to find the right audience.

3 Likes

I don’t think OCaml should try to be Perl. Other languages seem to do just fine with no regex support other than bindings. Features like raw strings or even special regex syntax do make them much more ergonomic in some languages. Does ppx_regex count ?

Perhaps the Python comparisons would be better, if we don’t focus on OCaml’s lack of dynamic typing or support for monkey patching. :smiley:

You can do that on www.rosettacode.org.
It’s also visible there that some tasks / algorythms fit better sometimes written with a functional style, and sometimes are better written with an imperative style.
You can propose new tasks, if it’s not already there.

2 Likes

I would put it differently: “OCaml’s verbosity and apparent requirement for the programmer to know vast numbers of module-names”

Viewed that way, there are things that could be done.

–chet–

Regarding Perl vs OCaml:
An (impressive) implementation of all the solutions of the Perl Cookbook
in the Objective CAML language (used at the time) is available here:
PLEAC-Objective CAML

Re-writing these examples with “modern” code/libraries could be very interesting.

6 Likes