How to manage deps between value/types and enforce referential integrity?

Let’s take the very daily situation of handling a set of OCaml modules (defined on a module per file basis or as modules within a module-file).

Case 1: only OCaml files
I would like to get a fine grain graph of all dependencies between OCaml components.
ocamldep gives all dependencies for a set of modules, in order to know in which order modules must be compiled.
How can I get a fine grain dependencies graph to the level of values (functions) and types?

Case 2: OCaml files referencing various files and directories (libs, static files, etc.)
Same question with OCaml files and referenced non OCaml files.

Last but not least:
how can you handle the referential integrity of values and types name?
e.g. in a basic web design tool, as soon as you modify the name or path of a file (or of piece of code), all pieces of code in all files that reference it are immediately updated.
In an OCaml file, the basic feature would be: if a value name is changed, then update all values that reference it.
First in the current buffer. Then, when the buffer is saved, in all values in all files that were using that value which name was modified (sync without saving could be an option).
This feature could be enabled/disabled, which would require to manually fix lost referential integrity, or to use a tool to find “dead references/links”, or to setup a meta referential that will keep track of references (established, lost) between values/types.

It’s one of my main concern when editing an Ocaml program that becomes large and distributed in many modules. Using the buffer memory or the (risky) multi-edition feature of the editor is not satisfactory. Even if type-safety protects us, this can be very time-consuming to enforce referential integrity that is anyway required by the compiler to do its job.

Is there any tool/package to do or build that when programming in OCaml?

Opam and Dune are obviously handling dependencies to get their job done, but I’m not enough aware of their internals to figure out how I could use them for that, or if it’s feasible.

Thanks.

Merlin has support for renaming (:MerlinRename in Vim) and I would assume it is used in Visual Studio Code as well. I don’t know how well it works across modules and interfaces.

Does :MerlinRename work at the scale of the buffer/file or at the scale of all your project files?

In vscode, C-F2 allows to edit multiple strings. I don’t know if it’s a native vscode or a specific Merlin feature.

:MerlinRename : “Rename all occurrences of identifier under cursor to .”

I don’t find any “rename” Merlin function in Emacs:

M-x merlin-
Possible completions are:
merlin-configuration-check 	
merlin-copy-enclosing
merlin-customize
merlin-debug-last-commands
merlin-destruct 	
merlin-destruct-enclosing
merlin-disable-debug 
merlin-document 	
merlin-enable-debug
merlin-enclosing-expand
merlin-error-check
merlin-error-next
merlin-error-next-in-group
merlin-error-prev
merlin-error-prev-in-group
merlin-error-reset 
merlin-extensions
merlin-find-file
merlin-flags
merlin-goto-project-file
merlin-jump
merlin-locate
merlin-locate-ident
merlin-mode
 merlin-occurrences
merlin-phrase-next 
merlin-phrase-prev
merlin-pop-stack
merlin-refactor-open
merlin-refactor-open-qualify
merlin-search 
merlin-setup
merlin-stop-server
merlin-switch-to-ml 
merlin-switch-to-mli 
merlin-toggle-view-errors
 merlin-type-enclosing
merlin-type-enclosing-go-down
merlin-type-enclosing-go-up
merlin-type-expr
merlin-use
merlin-version

When I did a quick experiment in Vim, :MerlinRename did not rename a value that was declared in an interface - this suggests it works on the level of the buffer and not the project even while knowing about it.

Ok.
Do you have any idea of a relevant way for doing that to the scale of all project files?

Regarding my goal, I imagine that I could use Merlin to crawl all over the project files and do the update to enforce referential integrity. Managing a referential of established/lost bindings would be too much complicated to start with.

Do you know a good way to do that?
Implement ing an Emacs module?
Implementing a Merlin plugin?
Maybe @let-def @gasche or any other maintainer mentioned on Github could make us a feedback about how to make such a Merlin plugin? (I don’t see that feature in the wiki documentation)

I wouldn’t be surprised if the conclusion was that OCaml is lacking in that department: we haven’t had a lot of tooling on those issues, historically. If you care about a certain feature which is not available, it’s a good idea to be a bit pushy as in this thread. Find out where it would fit best, ask the maintainers of the relevant part whether the feature could be implemented, and then maybe try to produce an implementation under their guidance.

That said, this type of feature may be available in the ROTOR refactoring tool, this is where I would go look for it / what I would play with to find out.

2 Likes

Thanks for that pointer to ROTOR that seem to have some of the features I talked about.
Some key OCaml language features are not (yet) supported (functors, first-class modules)

What motivated this topic is that I’m continuously discovering OCaml features sometimes used on a daily basis by experimented OCamlers. This means that I’m not really up-to-date, while new advanced features/concepts are approaching (Modular implicits, Effect type system…).
Before possibly improving something, the first thing is to fully understand what is existing, why it exists, use it, then make a motivated opinion about how a possible path. Hence my question.

I’m currently using basic Merlin features (types, jump to definitions, errors, auto completion). As the code base increases, I think that being able to do some kind of refactoring may be a major need.
Multi-line editing in a buffer is nice but far from being satisfactory (it doesn’t scale).
How do people working on large libraries and programs handle that requirement?

Necessarily, Merlin/Dune/Opam/OCaml are quite intricated (which is especially difficult for beginners that need to understand that the merlin file is generated by Dune which share knowledge about the program with Opam). So my question is probably not that easy to answer.

Who are the maintainers I should contact to gain visibility on the present situation, on a possibly shared desirable goal and about feasibility?

I would try to get in touch with the ROTOR people, and/or the Merlin maintainers, or both. It would be helpful to have specific concrete use-cases/examples to show (a minimal project with a proposed refactoring and the intended result); people can try each use-case in their tool and report on whether it satisfies that requirement.

1 Like

A basic text editor is sufficient to make a program with any PL. Especially for very experimented programmers.
But it quickly appears that some toolings can bring productivity and safety (jump to definition, autocompletion, project files explorer, verification tools, etc.)

I propose an informal QUICK POLL about the discussed feature
"Get referential integrity of an Ocaml program by getting an automatic (or proposed) update of all references to a value when its name is modified "

Definitions:
__ let foo = expr
__ foo is a name
__ expr is a value (an Ocaml expression)
__ let is the key word used to bind the name foo to the value expr

(* Initial situation *)
    let foo = (* some stuff *)

    let f x = 
      match foo x with 
        | Some x -> () (* some code *) 
        | None -> () (* some code *)
(* and many other functions using foo in the same file or in other files 
   that are parts of the OCaml program*)

When foo is renamed to bar AND the focus is put elsewhere (cursor position), the tool should automatically turn all previous references to foo into references to bar (or should propose to modify, depending on personal choice).

(* Final situation *)
    let bar = (* some stuff *)

    let f x =
     match bar x with 
        | Some x -> () (* some code *) 
        | None -> () (* some code *)
(* and many other functions are now using bar instead of foo *)
  • Today, how do you (Ocamlers) proceed to maintain referential integrity in your Ocaml program when you change the name of a value?
    – by hand ?
    – with a tool: which one?
    – to which scale? (file, project files)

  • Do you think the discussed additional feature is for you of:
    – null interest?
    – medium interest?
    – high interest?