C callbacks for GNU readline command completion

Learning
,
1

I’m having a OCaml program that calls GNU readline library via C interface.
For command completion, readline has a C variable rl_attempted_completion_function
which requires to be set as a pointer to a C function that the first argument is the string to be completed and the return value is an array of strings.

char **command_completion(const char *, int, int);

Then the readline() inside caml_readline will call rl_attempted_completion_function.

My problem seems to be:
Is there a way that I can manually free the char** returned by command_completion, which is converted from an OCaml callback function, or let it managed by the OCaml GC to free it after caml_readline has returned? Apparently the C caller to rl_attempted_completion_function won’t free the result for me (?*). Apologies for asking a more like C related question, but I have only too limit amount of knowledge to decide when to free allocated memories.

*: I’m not sure about that.

external caml_readline : string -> string option = "caml_readline"

/*
 * Read a line into a string buffer.
 * Returns a string option, None at EOF.
 */
value caml_readline(value prompt_arg) {

   CAMLparam1(prompt_arg);
   CAMLlocal2(v, b);
   char *line;

   line = readline(String_val(prompt_arg));

   /* Readline returns null on EOF */
   if(line == NULL) {
      /* None */
      CAMLreturn(Val_int(0));
   }

   /* This (probably) copies the line */
   if(line != NULL && *line != '\0') {
      /* Add nonempty lines to the history */
      add_history(line);
   }

   /* Copy the line */
   v = copy_string(line);

   /* Some v */
   b = alloc(1, 0);
   Field(b, 0) = v;

   /* Free the buffer */
   free(line);

   CAMLreturn(b);

}
2

If memory serves, you should return a freshly malloc-ed array of freshly malloc-ed strings (you signal the end with a NULL pointer) and the library will free the array after it is done with them, so no need to do anything fancy here.

Cheers,
Nicolas

1 Like
3

Thank you for the reply.

Does caml_string_length(s) includes the null end? That is, I’m having the following code converting the result, am I doing this right?

From caml_string_is_c_safe I’d assume caml_string_length(s) does not count the null byte.

static char **command_completion(const char *text, int start, int end) {

   char **matches = NULL;
   value words;

   /* disable file pathname completion */
   rl_attempted_completion_over = 1;
   if(start == 0) {
       words = caml_callback(*command_callback_f, caml_copy_string(text));

       int n = Wosize_val(words);
       if (n == 0) return(matches);

       matches = malloc((1 + n) * sizeof(char*));
       if (matches == NULL) return(matches);
       int i;
       for (i = 0; i < n; ++i) {
           value s = Field(words, i);
           char *cs = malloc(caml_string_length(s));
           matches[i] = strcpy(cs, String_val(s));
       }
       matches[n] = NULL;
   }

   return(matches);

}
4

OCaml strings consist of arbitrary sequences of bytes and may contain embedded NULLs. In particular they are not terminated with NULL, so you will need to add those by hand on the C side.

This is unsafe: you need to register the result of caml_copy_string as a GC root before calling the OCaml callback. Otherwise if the GC runs before you use that value (inside the callback), the value may be moved in the OCaml heap but the pointer won’t be updated and it will be left dangling.

As mentioned here you need to make space for the trailing NULL, so you must allocate caml_string_length(s) + 1 bytes, and instead of using strcopy you need to use something like memcpy.

Hope it helps,
Nicolas

1 Like
5

This is slightly misleading: the actual representation of OCaml strings in memory ensures that there is a NULL character just after the last character that is part of the string.
So as long as your string doesn’t contain other NULL characters (which you can check with caml_string_is_c_safe), you can cast it to a C string with the String_val macro.

1 Like
6

Indeed @vlaviron! I forgot for a moment this clever point about the encoding of strings in OCaml. Thanks! The OP still needs to reserve an extra byte for the trailing NULL though :slight_smile:

Best wishes,
Nicolas

7

If you are more interested in the command completion rather than the readline integration, you can have a look at linenoise

1 Like
8

Thank you for the feedback.

Does that mean declare the result using CAMLlocal and call CAMLdrop before return?

After read the caml/memory.h headers I think it is so. Incidentally, I find that the random crashing in the other http server library I’m using could also be caused by not playing nice with OCaml GC in C code.

9

Thank you for mention it, I’m actually more like to take this as an exercise learning how to interpolate OCaml with C, but this could be an implementation reference.

10

Indeed, you use CAMLlocal to declare the result, and, typically CAMLreturn and the end of the function (CAMLdrop is a low-level macro which is not documented if I’m not wrong).

I should have mentioned this before, but if you are writing C bindings you should read the official docs OCaml - Interfacing C with OCaml which are full of useful information.

Cheers,
Nicolas

1 Like
11

I don’t know for sure whether CAMLdrop is documented, but it works, doesn’t it? I used it with CAMLparam etc., in a C++ graphic interface that provides many, many virtual functions of which few will actually be used by an appl ication - I don’t want to go through all the motions to prepare for GC, if there’s actually no OCaml callback. So …

if (vfn) {
     ... acquire runtime (multi-threaded)
     CAMLparam0();
     CAMLlocalN(argv, 2);
     argv[0] = caml_alloc_custom(...); ...
     argv[2] = caml_alloc_custom(...); ...
     caml_callbackN(vfn, 2, argv);
     CAMLdrop;
      ... release runtime
}

I’m not saying this is proven to work - on the contrary, there’s a fair amount of crashes with trashed memory - but there’s a lot going on, so I have no particular reason to suspect this particular code. Do I?