Best way to access files in `lib`?

Learning
1

Heya!

I’m building a library that relies on additional text files to load data. I managed to set up Dune so that the data folder gets installed in the lib folder at build time by adding this to my dune file:

(install
 (section lib)
 (source_trees data))

However, I’m not sure about how to retrieve these files in my code. ChatGPT suggested using Findlib to get the path to the package folder.

let data_path filename =
  Filename.concat (Findlib.package_directory "faker") "data/" ^ filename

which is fine, but that has a big caveat: it only works when the library is actually installed as a package. If I only build the package, but don’t install it, then Findlib fails with

Fatal error: exception Fl_package_base.No_such_package("faker", "")

Is there a different approach that’s more robust to retrieve a path to a directory in lib?

2

Hello @Richard-Degenne

If the text data files are “part of the code” one option is use the preprocessor blob by adding to you dune (in lib) something like:

 (preprocess (pps ppx_blob))
 (preprocessor_deps (glob_files sources_txt/*.txt))

Then in your code you can access to the text with

let text1 = [%blob "lib/sources_txt/text1.txt"]
let text2 = [%blob "lib/sources_txt/text2.txt"]

Note that this means that your text files will be included in the binary/executable.

Other option within dune is described here: How to Load Additional Files at Runtime - Dune documentation

1 Like
3

As you point out, I would like to avoid bundling the text files in the binaries.

However, the dune_site option sounds exactly like what I was looking for. I’ll give that a shot.

Do you know if the warning about it being experimental is still relevant?

WARNING: This feature remains experimental and is subject to breaking changes without warning. It must be explicitly enabled in the dune-project file with (using dune_site 0.1)

4

Do you know if the warning about it being experimental is still relevant?

In the source of dune the dune-site feature is still there:

so I think that it should work. I hope the examples of the documentation are still fine.

5

I’m making good progress using dune-site: it does exactly what it says on the tin.

However, I’m running into a bit on an issue: the files are only copied when running dune build @install.

I would like the copy to run as part of the dune test pipeline, since my unit tests depend on the files being present. I have added this stanza to my test/dune file, but I don’t know if this is proper.

(alias
 (name runtest)
 (deps
  (alias ../install)))
6

I think you need to do something like:

(test
  (name foo)
  (deps (package yourpackage)) ;; <- THIS
)

To get them available during testing.