Is there a (not too complex) way to have jbuilder successfully build a small project relying on ppx_deriving?
“not too complex”: because I’ll do it with ocamlbuild if it appears to be simpler
I tried to add (preprocess (pps (ppx_deriving.enum))) to my library, but when compiling the relevant module, the functions that should be generated by ppx_deriving are not found.
Thanks for any tip
As far as I can tell - no. I gave it a go to make ppx_deriving work with jbuilder a few weeks back https://github.com/whitequark/ppx_deriving/pull/135. However, due to time constraints I had to abandon my attempt. I am guessing ppx_deriving plugin based design + omp has something to do with it.
There have been a few attempts. Currently it seems that you only need to wait for this to be merged: https://github.com/whitequark/ppx_deriving/pull/141
I’d like to use ppx_deriving_yojson with jbuilder. I’ve tried pinning ppx_deriving to ppx_deriving#141 as suggested, but my project still fails to compile:
Error: Attribute `deriving' was not used
Has anybody successfully used ppx_deriving with jbuilder? Would greatly appreciate more details on how to get jbuilder working with ppx_deriving_yojson (or another ppx_deriving-plugin).
I think ppx_derivign_yojson also needs to be ported to ocaml-migrate-parsetree (on the lines of https://github.com/whitequark/ppx_deriving_yojson/pull/49 ± adaptations), but I might be wrong.
I’m running into something which I think is an instance of this issue, but I’m not sure. I’m using a jbuild file which has (preprocess (pps (ppx_jane ppx_deriving.std))), but when running jbuilder build @install I get errors such as the following for eq and enum:
ppx lib/Frenetic_NetKAT.pp.ml (exit 1)
(cd _build/default && ./.ppx/ppx_jane+ppx_deriving.std/ppx.exe --dump-ast --cookie 'library-name="frenetic"' -o lib/Frenetic_NetKAT.pp.ml --impl lib/Frenetic_NetKAT.ml)
File "lib/Frenetic_NetKAT.ml", line 8, characters 55-72:
Error: ppx_type_conv: 'eq' is not a supported type type-conv generator
Is this the same issue as previously described in the thread?
If you want to use ppx_deriving in the mean time, with pds, one can do deps = ["ppx_deriving.blah"] or deps = ["ppx_deriving_yojson"], although it applies to a whole project rather than just to specific source files.
I wanted the same: ppx_deriving works with jbuilder now, but you’ll need ppx_deriving_yojson with this change (pin or wait for it to get merged/released) ppx_deriving_yojson#60. With that, it works now!
My library has been compilable for months with (preprocess (pps (ppx_deriving.enum))) in the jbuild file.
All of a sudden, it stopped working, either because of an update or because I tried to switch to dune?
# File "lib/dune", line 6, characters 13-36:
# Error: No ppx driver were found.
# Hint: Try upgrading or reinstalling ocaml-migrate-parsetree.
But it seems that the last version is already there:
$ opam show ocaml-migrate-parsetree
=-=- ocaml-migrate-parsetree: information on all versions -=-=-=-=-=-=-=-=-=-=-=
name ocaml-migrate-parsetree
all-installed-versions 1.0.11 [4.06.1+flambda]
all-versions 0.6 0.7 1.0 1.0.1 1.0.2 1.0.3 1.0.4 1.0.5 1.0.6 1.0.7 1.0.8 1.0.9 1.0.10 1.0.11
=-=- Version-specific details -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
version 1.0.11
repository default
url.src: https://github.com/ocaml-ppx/ocaml-migrate-parsetree/releases/download/v1.0.11/ocaml-migrate-parsetree-1.0.11.tbz
url.checksum: md5=26bb1b038de81a79d43ed95c282b2b71
homepage: https://github.com/ocaml-ppx/ocaml-migrate-parsetree
bug-reports: https://github.com/ocaml-ppx/ocaml-migrate-parsetree/issues
dev-repo: git://github.com/ocaml-ppx/ocaml-migrate-parsetree.git
authors: "Frédéric Bour <frederic.bour@lakaban.net>"
"Jérémie Dimino <jeremie@dimino.org>"
maintainer: frederic.bour@lakaban.net
license: LGPL-2.1
tags: "syntax" "org:ocamllabs"
depends: "ocaml" {>= "4.02.0"}
"result"
"ocamlfind" {build}
"jbuilder" {build & >= "1.0+beta18.1"}
synopsis Convert OCaml parsetrees between different versions
description This library converts parsetrees, outcometree and ast mappers between different OCaml versions.
High-level functions help making PPX rewriters independent of a compiler version.
Did I miss something obvious?
Mmm. It works again after I started a fresh opam switch.
I’m going to try to find a way to “clean” my usual switch.
Sorry for the noise.
@ttamttam I refactored the handling of ppx rewriters in dune. Before the knowledge of a few ppx projects and their behavior was hardcoded in jbuilder. It wasn’t great as it made the whole system quite rigid. In particular new features had to be synchronized between jbuilder, ocaml-migrate-parsetree, ppxlib and ppx_driver which was a lot of hassle.
With the new system, dune has a generic support for ppx drivers and projects such as ocaml-migrate-parsetree or ppxlib simply declare themselves as ppx drivers. In order to ensure backward compatibility, this new system is only activated when you use a dune file. When using a jbuild file it is still the old system with hardcoded values.
Now, in order to use the new system, ocaml-migrate-parsetree and ppxlib must declare themselves as ppx drivers. The released versions of ocaml-migrate-parsetree and ppxlib already have the relevant information in their jbuild file to declare themselves as ppx drivers. However, only dune will read this information and in particular copy it to the installation directory. This means that if ocaml-migrate-parsetree was build and installed with jbuilder <= 1.0+beta20, then dune cannot see that ocaml-migrate-parsetree is a driver.
Forcing a resintallation of ocaml-migrate-parsetree via opam reinstall ocaml-migrate-parsetree will fix the issue.
Thanks for the explanation!
Maybe there should be a message after upgrades to dune to do the reinstall omp in presence of ppx errors, as this seems to affect lots of people
See above. The message is pretty clear. I just should have done what was written: upgrade or reinstall. Maybe, it can be improved by quoting the two commands to be tried (at the moment, I did not realize opam had a ‘reinstall’ sub-command).
Thinking about this again, making ocaml-migrate-parsetree and ppxlib explicitly depend on dune in opam should fix this issue. I’m not in front of a computer today but if someone could do it that would be cool 
A few versions of these two packages are marked as incompatible with dune, but the latest versions of both are compatible with dune. Changing the dependency from jbuilder to dune for these latter ones should do.
While that is fine, this is going to cause a super massive amount of recompilation for downstream users. As virtually any package is going to depend on ppxlib and omp. I suppose that it can’t be helped and that the correctness is far more important here. What would be nice if opam was somewhat smarter about reinstallation. If the builds are deterministic it should be able to trigger downstream reinstall for cases when the artifacts for omp/ppxlib are going to differ.
PS: ppx_inline_test should need a similar change as well then.
Unfortunately there is not much opam can do here: when build ant installed by dune, the ocaml-migrate-parsetree.dune file is different compared to when built and installed by jbuilder. opam cannot know whether this change matters or not for reverse dependencies of omp so it will always need to rebuild them.
Only Dune can do something here: once we’ll have clould builds, we could enable a shared artefact cache for opam builds and then these reinstallations will be really fast.
The rebuilds are a bit unfortunate but in the end it’s not much different from a new release of omp. I made a PR: https://github.com/ocaml/opam-repository/pull/12350
How does one use ppx_deriving with dune? I tried the following:
(executable
(name test)
(preprocess (pps (ppx_deriving.show))))
But dune complains:
File "dune", line 3, characters 18-37:
3 | (preprocess (pps (ppx_deriving.show))))
^^^^^^^^^^^^^^^^^^^
Error: Atom or quoted string expected
Removing the parentheses doesn’t help:
$ dune build test.exe
File "dune", line 3, characters 13-36:
3 | (preprocess (pps ppx_deriving.show)))
^^^^^^^^^^^^^^^^^^^^^^^
Error: No ppx driver were found. It seems that ppx_deriving.show is not
compatible with Dune. Examples of ppx rewriters that are compatible with Dune
are ones using ocaml-migrate-parsetree, ppxlib or ppx_driver.
What’s the trick?
Notice that the second error is not the same as the first one. The second snippet is correct, and should work, but maybe your version of ppx_deriving is an oldish one? I think support for dune was added last november or so.
I think I’m using the latest version of ppx_deriving for 4.03.0
$ opam show ppx_deriving
<><> ppx_deriving: information on all versions ><><><><><><><><><><><><><><><><>
name ppx_deriving
all-installed-versions 4.1.5 [4.03.0] 4.2.1 [4.04.2]
all-versions 0.1 0.2 0.3 1.0 1.1 2.0 2.1 2.2 3.0 3.1 3.2 3.3 4.0 4.1 4.1.5 4.2 4.2.1
<><> Version-specific details <><><><><><><><><><><><><><><><><><><><><><><><><>
version 4.1.5
repository default
url.src: "https://github.com/whitequark/ppx_deriving/archive/v4.1.5.tar.gz"
url.checksum: "md5=8112a61f00ae5c38bf3bbbfde3263a85"
homepage: "https://github.com/whitequark/ppx_deriving"
bug-reports: "https://github.com/whitequark/ppx_deriving/issues"
dev-repo: "git+https://github.com/whitequark/ppx_deriving.git"
authors: "whitequark <whitequark@whitequark.org>"
maintainer: "whitequark <whitequark@whitequark.org>"
license: "MIT"
tags: "syntax"
depends: "ocaml" {>= "4.02.1"}
"ocamlbuild" {build}
"ocamlfind" {build & >= "1.6.0"}
"cppo" {build}
"cppo_ocamlbuild" {build}
"ppx_tools" {>= "4.02.3"}
"result"
"ounit" {with-test}
synopsis Type-driven code generation for OCaml >=4.02
description ppx_deriving provides common infrastructure for generating
code based on type definitions, and a set of useful plugins
for common tasks.
… which probably means that my compiler is too old. Thanks for the pointer; it works in 4.05.0.