Jupyter with pyml

Community
,
#1

Hi, I would like to achieve something similar to, but different from what is discussed in OCaml in Jupyter-based Environments . tl; dr: How can I get python REPL access to OCaml values exported via pyml?

The use case is this: I have an OCaml library that does machine-learning type computation. I would like to generate, plot and analyze the results interactively, with the best possible ease of use and flexibility. For the plotting and analysis I want to use Python.

I have tried ocaml-jupyter, which provides a nice notebook environment but no python kernel and the very useful ocaml-matplotlib which offers a matplotlib binding but cannot cover all my python library needs.

A possible solution would be to use pyml and its numpy array memmapping support to export OCaml values to a python process. The problem here is that I can’t find a way to interact with the python process spawned from pyml in an interactive python console. I am thinking there should be some way to spawn a Jupyter python kernel from pyml and then access that from a separate jupyter console. I have not been able to find one by googling though.

Another solution might be to wrap my OCaml library as an external python module. I’m afraid that this may be painful – please convince me otherwise :wink:

2 Likes
#2

Hi @n4323,

You may be interested in this post on the Jane Street tech blog:

Using Python and OCaml in the same Jupyter notebook, by Laurent Mazare (@laurent).

which offers some general tips and some bespoke tooling (showcased in https://colab.research.google.com/drive/1MDuZc0v60lzBg0_xeiPR-bB6dCIWxvzT). I have found this very useful in the past when trying to do Python + OCaml interop in Jupyter notebooks.

4 Likes
#3

Cool, that looks like it would help a lot with the second option of wrapping an OCaml library. I’ll definitely check that out.

Edit: after checking it out it seems that this may be best solution. From the example dune file I think that wrapped OCaml libraries are compiled to native code; still I guess the ocaml in %%ocaml magic cells is run as bytecode?

#4

OCaml libraries complied in byte code and in native code should both be usable from Python. If I remember correctly the reason for the toplevel library to be compiled as bytecode is that the native toplevel is not exposed in compiler-libs (if it was exposed this should work when compiled as native).

1 Like
#5

I know it is very late, buy anyways

You should be able to write code in different languages and treansfer generated data between those code segments within Emacs Org-mode Babel extension. It works kind of like Jupyter, but more flexible when it comes to combine different languages.

There even papers written in Org-mode with code that is evaluated when the PDF file is generated (if one doesn’t want to use the cahced previous run).