One common functionality that I’ve found myself wanting to incorporate into applications that I build is the support for user-defined plugins/extensions (without having to re-compile the application itself), but I don’t know what is the recommended way of providing a plugin system for OCaml programs?
Now obviously one approach would be to define an extension language and include a parser/interpreter as part of the application, but this is a hassle, and introduces the additional burden of having to provide language support/documentation.
Is there a clean interface for using OCaml itself as an extension language? i.e the user writes OCaml code, compiles it to bytecode/native-object-code, and then loads it into the main OCaml program? Are there any good examples of this being done in the wild?
Depends on your requirements, but personally I generally prefer plugins to be separate executables that use IPC. It is better security and lets users write plugins in any language. The downsides is you have to define a protocol over IPC (lots of options available) and maybe it’s too heavy weight for what you want to do.
The basic tool is Dynlink which allows to load OCaml code at runtime and link it into the main program. The plugin is supposed to “register” itself with the main program at initialization time, as the main program has no way to directly call into the plugin code.
A tricky point is how to deal with dependencies among plugins (in the case where plugins can depend on each other). For that there is findlib.dynlink, and Dune offers a higher-level (experimental) interface called “sites”, see How to load additional files at runtime — dune documentation.
The kind of picture I’m going for is that the user installs the application once, and then may extend the functionality by adding plugins on a per usage basis (probably via command line arguments).
Maybe, but this also makes sharing data between the main application and plugins to be a hassle, requiring the input to be serialised. Ideally I’d want to provide the user a module-signature interface that they can implement in OCaml however they want, rather than having to go through IPC.
Perfect, thanks! This looks like what I was looking for. ocaml_plugin seems to load in the OCaml compiler as well for compiling source files which is useful, but I guess I can settle with just Dynlink. The dune-sites functionality seems interesting, I’ll need to have a look.
In general, toplevels (ocaml or ocamlnat) do not know anything about dependencies (that’s one of the reasons for having findlib). As for the ability to link native-code they work roughly the same as Dynlink.
What I’ve done is provides a little framework that does the IPC for them and then calls into a function they provide. And, arguably, making sharing data harder is a benefit to security. But I take your point that you’d prefer in-process plugin.
As @nojb said, Dynlink is a suitable tool for this feature.
If plugins need ad hoc polymorphism, the natural way is to use OOP (this is what I do in Graffophone). However, It may be possible to use Kantian’s solution.