How to expose a function to a _db.ml file in ocsigen?

ruyblast · April 17, 2020, 3:48pm

Hi !
Writing some pgocaml/database related code in a _db.ml file (as required), I realized that I had difficulties exposing a simple function from another file. Here is what I tried:

(* inside a function in a _db.ml file: *)
let hstore_opt = Utils.one_of_opt_db hstores_opt in
...

(* inside utils.eliom: *)
[%%shared
let filter_none_db l =
  let rec skip_none acc_l = function
    | None :: tl -> skip_none acc_l tl
    | Some e :: tl -> skip_none (e::acc_l) tl
    | [] -> acc_l
  in
  skip_none [] l
    
let one_of_opt_db l =
    match filter_none_db l with
    | a :: _ -> Some a
    | [] -> None
]

It lead me to following error message:

File "administrative_book_db.ml", line 240, characters 19-38:
240 |   let hstore_opt = Utils.one_of_opt_db hstores_opt in
                         ^^^^^^^^^^^^^^^^^^^
Error: Unbound value Utils.one_of_opt_db
Makefile.os:235: recipe for target '_server/administrative_book_db.cmo' failed

And it is the same if I replace [%%shared ... ] by [%%server ... ] (and even if I had the same declarations in a [%%client ... ] block just after).

It is very surprising to me for three reasons:

1/ as in ocsigen-start demo_pgocaml_db.ml, this file starts with open Os_db and uses function full_transaction_block without any problem.

2/ I expose a lot of types declared in shared section from another module without problem

3/ I exposed already a log function from utils.eliom. This function is a bit special, I wrote it this way so that I could called it the same way from client and server side:

let%server log s =
  let s = " # log: "^s in
  let%lwt _ = Lwt_unix.write_string
         Lwt_unix.stdout
         (s)
         0
         (String.length s)
  in
  Lwt.return ()

let%server log_from_client =
  Eliom_client.server_function
    [%json: string]
    (Os_session.Opt.connected_rpc (fun userid s -> log s))

let%client log_from_client = ~%log_from_client

let%client log = ~%log_from_client

(I tried to expose one_of_opt_db the same way with an rpc, but I could not serialize polymorphic type 'a option list to json)
I can of course simply write those functions inside the _db.ml file, but I find it a bit frustrating to do this way if I want to split this file later.
What do you think I am doing wrong?
(I use eliom 6.11.0 and ocsigen-start 2.16.1)
Thanks a lot for your help!

Zzull · April 23, 2020, 2:31pm

Hello,

As you already found out, there’s nothing preventing you from having a _db.ml file depending on an .eliom file. That being said, it might be the case that utils.eliom depends, in some indirect way, on your _db.ml file. If this is the case, you can confirm that by taking a look at the .depend file (lying in the root directory of project) or putting the two functions that you exposed in a new .eliom file.

Hope this helps

ruyblast · April 28, 2020, 4:16pm

Hello @Zzull,
I think I spotted a bug in ocsigen-start build system thanks to your help. Here I explain my investigation:
In the .depend file, I could see that my _db.ml file and my utils.eliom file was not depending on each other:

_server/administrative_book_db.cmo : _server/type.cmo
_server/administrative_book_db.cmx : _server/type.cmx
...
_client/utils.cmo :
_client/utils.cmx :
_client/utils.cmo : _server/utils.type_mli
_client/utils.cmx : _server/utils.type_mli
...
_server/utils.cmo :
_server/utils.cmx :
_server/utils.cmo : _server/utils.type_mli
_server/utils.cmx : _server/utils.type_mli

I was not surprised that utils.eliom file did not depend on _db.ml file, but the opposite surprised me a lot because I make calls to Utils.log inside _db.ml file without any compile time nor runtime problem.
I decided to move definitions of filter_none_db and one_of_opt_db into another file (utils_bis.eliom), and I got a compiler error:

File "administrative_book_db.ml", line 367, characters 12-35:
367 |   let s_l = Utils_bis.filter_none_db s_l_o in
                  ^^^^^^^^^^^^^^^^^^^^^^^^
Error: Unbound module Utils_bis

So I deduced that make test.byte does not automatically make aware a _db.ml file of other modules.
I tried a local open let open Util_bis in, but it did not change anything.
As I import values from a module called Type (from type.eliom) with a global opening at the beginning of the file, I tried a global open open Utils_bis at the beginning, and then it worked!

Although the problem was solved, I wanted to understand why I could open only part on Utils module into _db.ml file. Here is my further investigation:
I checked .depend file. It stated that _db_file was now depending on utils_bis.eliom. Then I removed open Utils_bis and the beginning of _db.ml file, I saved, I compiled, and… everything was fine: no compile time nor runtime error while I was using Utils_bis module. I checked .depend file that stated that _db.ml file did not depend on utils_bis.eliom anymore.

I began to understand that .depend file for _db.ml file keeps only track of dependencies for modules that are globally opened. Then if a global open is removed, dependency is removed from .depend, but previously built interface for _db.ml file remain (that is why I could import an “old” log function from utils.eliom, but not a new one).

To confirm this hypothesis, I added a small function to utils_bis.eliom:

let%shared do_nothing () = ()

and I tried to call it inside my _db.ml.eliom:

let () = Utils_bis.do_nothing () in

And I got error:

Error: Unbound value Utils_bis.do_nothing

That was solved again by a open Utils_bis at the beginning of _db.ml file, that I could remove after one succesful compilation.

So to sum it up:

a _db.ml file cannot be aware of values in another .eliom file without a (possibly removed in the past) global open.
there is a problem of synchronization between .depend file and built interface that expose modules to others (“real dependencies”) for _db.ml files.

Again thanks for your help,
Have a good day !

[EDIT : I could not reproduce the bug on a fresh ocsigen-start project, so I did not open an issue about it on github]

Zzull · April 29, 2020, 12:35pm

You definitely put your finger on something fishy in Makefile.os. And your answer made me remember that I actually did stumble on something similar too. And the ‘solution’ (which isn’t one because it’s not fixing the Makefile) was to make distclean the project.

ruyblast · April 30, 2020, 1:26pm

Thank you @Zzull for sharing your tip ! I just tried make distclean but it did not remove the strange behavior. For now I am ok with the “fix” I found, and maybe one day I will explore Makefile.os.

Best !