Can we improve type error messages?

[octachron]

@Release-Candidate

• why format4 when there are just 3 arguments?

? format4 takes four argument?

@Release-Candidate

• why is there a format6 mentioned? Where does that come from?
• what does format6 have to do with format4? How should somebody know, that format4 is the same as the 3rd argument used thrice in format6 (IIRC) in this case

I agree with that that implicitly following type aliases is confusing here. I am inclined to think that we want to keep the initial type in the error message, another option would be to also point to a type declaration that exposes the constructor but that feels noisy.

@Release-Candidate

most newbies don’t know that :: is a constructor (too), they think that is is (only) a infix operator. So there is no connection to a list.

So you agree that the error message should remind users that the error message might be due to the list literal?

@Release-Candidate

• the carets to not point to the opening bracket [

Good point, I didn’t notice that the location is slightly wrong here, thanks !

@Release-Candidate

“Error: This variant expression is expected to have SOME OTHER TYPE” with no idea how that type should look like

Are you arguing that the error message should list some of the constructor of the SOME OTHER TYPE type?

@Release-Candidate

The message would be less confusing if it just pointed at the variant expression and said something like “There is something wrong!”

That seems blatantly false since you said yourself that the error message is stating that "“Error: This variant expression is expected to have SOME OTHER TYPE” .

[yawaramin]

@gasche

I’m not familiar with Scala3 error messages, can you show or point at relevant examples?

Sure. Here’s an example adapted from earlier in the thread:

enum S:
  case Ax
enum T:
  case Ax
enum U:
  case Ax
enum V:
  case A

def f(s: S, t: T, u: U, v: V) = ()

To contrast with OCaml, Scala doesn’t have type-directed disambiguation. You have to specify the enum container of the case constructor. Now let’s trigger an error:

-- [E007] Type Mismatch Error: -------------------------------------------------
1 |f(S.Ax, T.Ax, U.Ax, U.Ax)
  |                    ^^^^
  |                    Found:    (U.Ax : U)
  |                    Required: V
  |
  | longer explanation available when compiling with `-explain`
1 error found

@octachron

I am sorry, but would you mind listening to what I have been explaining to you since the start of this strange discussion?

Let’s tone this down a little. I understand you are frustrated but I promise you I am not doing this on purpose, it’s a misunderstanding (on my part, for which, I apologize).

This is what happens today in OCaml typechecker. This is a fact, and not a point that you can discuss.

OK, understood. My mental model, wherein I thought that the types of all parts of the program were inferred and then checked against each other, was incorrect. Instead here it’s just searching for constructor Ax in the required type v and not finding it.

I believe that you would better spend your time and energy explaining what you find so confusing in the initial error message.

OK. My confusion is that I expect a straightforward type error here, but get an indirect reference to the fact that type-directed disambiguation was attempted (and that type inference was actually not attempted). If type inference were done, it would be immediately obvious (in this case) that we are passing a list instead of the required format string.

So let’s try to move forward with what we have. How about this instead:

File "./test.ml", line 6, characters 23-25:
6 | let error = f Ax Ax Ax Ax
                           ^^
Error: expected value of type
  v
But got constructor
  Ax

And in the case of the list:

File "bin/main.ml", line 292, characters 53-70:
292 |       textarea [ id "notescontrol"; name "notes" ] [ txt "%s" c.notes];
                                                           ^^^^^^^^^^^^^^^^^
Error: expected value of type
  ('a, unit, string, node) format4
But got constructor
  ( :: )
Hint: you may be accidentally passing a list.

In case you are wondering–yes, I am also suggesting trimming the message down a bit. In daily use we quickly learn to skim the error messages and focus on the most relevant parts. If helps if they are structured in a way that is amenable to quick scanning.

EDIT: slightly tweaked message for clarity.

[octachron]

@yawaramin

Let’s tone this down a little. I understand you are frustrated but I promise you I am not doing this on purpose, it’s a misunderstanding (on my part, for which, I apologize).

And sorry for getting frustrated because I failed to communicate! Thanks a lot for taking the time for answering in a more composed way than me.

@yawaramin

So let’s try to move forward with what we have. How about this instead:

File "./test.ml", line 6, characters 23-25:
6 | let error = f Ax Ax Ax Ax
                           ^^
Error: expected type
  v
But got constructor
  Ax

I am totally open to discuss redesigning the global format for mismatches; and I have a plan to allow easy experimentation in this direction for anyone without having to patch the compiler; but that’s a quite a huge and separate matter.

If we stick to the current template “This is … but we expected …”, so that we can have a first step improvement in 5.2, I would reword your proposition to something like

File "./test.ml", line 6, characters 23-25:
6 | let error = f Ax Ax Ax Ax
                           ^^
Error: This variant constructor Ax does not match the expected type v

what do you think?
Similarly, I would amend your hint to

Hint: you may be accidentally using a list literal.

since the error message can also appear in pattern position.

[yawaramin]

Yes, I like both these suggestions as a good first step. I think they will alleviate some newcomer confusion.

Regarding the bigger matter of error message redesign, I am actually very interested in that too. I was experimenting a bit with it late last year, piggybacking on top of Eduardo’s coloured type errors PR but exploring structuring with indentation instead. It does look a bit tricky. I’ll try experimenting with it a bit more on my end.

Thanks for engaging.

[Release-Candidate]

@octachron

? format4 takes four argument?

Yes, but in this case two of them are the same (IIRC the last one).

@octachron

@Release-Candidate

The message would be less confusing if it just pointed at the variant expression and said something like “There is something wrong!”

That seems blatantly false since you said yourself that the error message is stating that "“Error: This variant expression is expected to have SOME OTHER TYPE” .

I was trying to explain to you that this part (in my experience) does not help people that don’t (yet) understand this kind of type related error messages. I’ not saying to not add this for people that do better understand these error messages from the unification process.

This post has just been made to show you want can be the problems for someone not used to reading OCaml (or similar languages) error messages.
So now some suggestions on how to better the error message (I should have added them in the original post):

• using the least amount of words possible. Adding “variant expression” does not help people who know what a variant expression is - they know that the error is there - and confuses people that don’t know about them, because now they are unsure if what is pointed by the carets to is the variant expression or the whole line or … Better just print out the erroneous expression again:

Error in "[ txt "%s" c.notes]"

• what helps is explicitly stating what type is expected and what is given instead. Do not use a phrase and always the same layout. Like

Expected: type "('a, unit, string, node) format4"
Actual: constructor "::", which is incompatible with type "format6"

• printing the whole desugared expression that generated the error, like

The constructor "::" in "(txt "%s" c.notes) :: []" ....

But of course changing the layout of the error messages breaks all error message parsers.

[R_Huxton]

As the original poster, perhaps I should expand on my original grumble. My reasoning is clearly not correct below, but it was (as far as I remember) my actual thought process.

Bear in mind I’ve got plenty of experience over the decades in various languages from 8 bit assemblers through all the normal ones to a bit of Smalltalk and Lisp and a smattering of Haskell. I’m used to debugging and I’m used to not always having the greatest of error messages.

Expected: type "('a, unit, string, node) format4"
Actual: constructor "::", which is incompatible with type "format6"

So - as it happens I knew that what this means (at my level of understanding) is “bad format string”. That’s because I’ve seen similar before and was reduced to the make-random-change-until-something-happens process to get past it. Then I figured out that 9 times out of 10 (for me) it’s a missing-semicolon situation.

So, I tried to be a good citizen and checked the source-code of the library:

  let textarea attrs fmt = text_tag "textarea" attrs fmt

So - that’s not hugely helpful because it’s clearly looking for a fmt which doesn’t match the error-message in any sense and also isn’t defined in the library. (I’m checking this on github if that matters).

OK - so let’s try and decode the type: format4 is clearly formatting related and since there are 4 parameters listed in front of it I can guess what the number might be.

The first two parameters: 'a unit - useless. Utterly without meaning. The first one clearly can be “anything” and the second one is… “unit”? This format4 expects 4 parameters and the second has to be the empty type - either the author is mad or I’m missing something fairly important here. (we all know it’s the second one, but what don’t I understand that would make this make sense)

OK - third parameter is string - doesn’t tell me much. It might be related to the actual "%s" stuff for formatting but it’s in the wrong place. If it isn’t representing the formatting string though, I’ve got no idea.

Last parameter-type node sounds like something HTML-y and indeed it is defined in the library! it’s a tag or text or comment - that makes sense.

OK - so the two key problems from the “expected” part of the error message are:

1. The type in the library I’m calling: fmt isn’t mentioned anywhere in the error.
2. The type that is mentioned requires elements that don’t seem to make sense to provide and actually just don’t make any sense at all at my level of understanding (“anything” and “nothing”).

For the “actual” part of the error message.

First time reading it I missed the implication of :: being list constructor altogether. Just read it as Constructor <punctuation> is incompatible.... Didn’t even pick up on what the punctuation was.

The bit I focused on at first was format6. Because one thing I certainly wasn’t trying to do was format 6 values or anything like that. Any way you cut it there was one “thing” involved. Either the list is the one thing or the fact that it has a single element is. Where the other 5 things are coming from - no idea.

Inside my list I have txt "%s" c.notes and here is the definition for the txt tag:

let txt ?(raw = false) fmt = ...

OK, so it takes an optional “raw” which I’m not supplying and… one parameter which is fmt.

That seems to be the same fmt which is identified with format4 in the first part of the error and is supposed to take 4 parameters of which two make no sense.

At this point my ability/willingness to drill any deeper is gone.

To summarise:

• We have a single parameter fmt in the library’s source code
• The compiler thinks that should actually be a complex type with 4 parameters of its own
• It thinks I’ve somehow provided a complex type with 6 parameters (or rather I haven’t done so properly)
• But it also thinks txt <format4> == <format6> in some way. Where the extra two parameters are coming from is unclear.
• Neither of the types mentioned in the error message are defined in the library’s .ml file
• The merlin type hints in vim just repeat what the compiler error is telling me (which is fair enough, if it had better errors than the compiler you’d just copy that code to the compiler)
• I don’t think the dream-html library is at any fault here - it seems like plainly written code and has docs and examples.

Now, like I said I knew that what this error actually meant was “you’ve messed up a format string” which usually means a missing semicolon or something. In this case it wasn’t just a semicolon and trying to actually understand the root cause got me nowhere.

Hopefully it’s just me going down the wrong path or using the wrong tools or something. Or maybe format-string handling is always going to be a complex piece of library code that generates complex errors and I got unlucky. I have to say that for my own code (with its frankly pedestrian types) the error messages pretty much just say exactly what I need to change.

HTH

[Release-Candidate]

@R_Huxton

• The compiler thinks that should actually be a complex type with 4 parameters of its own
• It thinks I’ve somehow provided a complex type with 6 parameters (or rather I haven’t done so properly)
• But it also thinks txt <format4> == <format6> in some way. Where the extra two parameters are coming from is unclear.

A way to get more information about types is using the REPL:

utop > #show format;;
type ('a, 'b, 'c) format = ('a, 'b, 'c, 'c) format4

utop > #show format4;;
type ('a, 'b, 'c, 'd) format4 = ('a, 'b, 'c, 'c, 'c, 'd) format6

utop > #show format6;;
type ('a, 'b, 'c, 'd, 'e, 'f) format6 = ('a, 'b, 'c, 'd, 'e, 'f) format6

There you can see, that the type format, which takes 3 arguments in the end is format6, like:

type ('a, 'b, 'c) format = ('a, 'b, 'c, 'c, 'c, 'c) format6

[SkySkimmer]

@R_Huxton

The type in the library I’m calling: fmt isn’t mentioned anywhere in the error.

I don’t understand this. Why would fmt be mentioned?
It’s the equivalent of x in

let f x = x + x

let _ = f ""
(* this expression has type string but an expression was expected of type int *)

[R_Huxton]

@SkySkimmer

I don’t understand this. Why would fmt be mentioned?
It’s the equivalent of x in

I probably explained myself badly, and like I said, I’m quit likely thinking about this badly. But there is a single parameter there: fmt and I went looking for a single type in the library and didn’t see anything.

What I’d sort-of expect is to see something along the lines of:

type text_tag_format = ...

let txt ?(raw = false) (fmt: text_tag_format) = ...

There’s a parameter - it’s a single thing. It has a type - that type has a name because it too is a single thing. I can look up that name and see what the type is supposed to be.

To give an example from “dream” itself - its docs page (and presumably its source - haven’t checked) has the following:

type request = client message

and response = server message

and handler = request -> response promise

and handler = request -> response promise

Those types get used all over the place in dream and they have names and even if I don’t know exactly how they work or what goes in them I can see when I’m supposed to supply them and if I’m getting it wrong.

If I’d had the idea to check what format4 was in utop like @ Release-Candidate suggested I’m not sure seeing the following would have helped at all

utop > #show format4;;
type ('a, 'b, 'c, 'd) format4 = ('a, 'b, 'c, 'c, 'c, 'd) format6

Four parameters that I don’t have - their types can be anything and none of them mean anything because they’ve clearly just been given single-letter names in alphabetical order. That’s less information than the actual error message.

So - the concrete problem in terms of “new ocaml dev dealing with error” is, starting from this:

Expected: type "('a, unit, string, node) format4"
Actual: constructor "::", which is incompatible with type "format6"

For this line of code:

textarea [ id "notescontrol"; name "notes" ] [ txt "%s" c.notes];

How do I come to an understanding of what actual values I need to type to provide 'a and unit and string and node. Because out of txt, "%s" and c.notes all I’m seeing is a function and two strings.

What, mechanically are the steps you would need to take to figure that out, and how many of them are there? I’m talking in terms of step-by-step instructions like you have for “how to install” or something. How do you locate the relevant definitions that explain what needs to be provided?

I think that would be the single most valuable thing to provide to newbies - “How to debug - a beginners guide”.

Because if we had that and past-me had worked out why the correct line is:

textarea [ id "notescontrol"; name "notes" ] "%s" c.notes;

I might also be able to understand why this didn’t:

textarea [ id "notescontrol"; name "notes" ] ("%s" c.notes);

since at the level of understanding I have, in the first one fmt = "%s" c.notes and in the second one fmt = ("%s" c.notes). If this was lisp, I’d understand but I thought avar = x and avar = (x) would be the same.

[soya]

Wrt. to types parameters without names.

The type parameters can be labelled by jumping through several hoops, requiring ocaml OO syntax, adding a type alias, and type parameter constraints.

Labelled type parameters in OCaml

Not sure it would help in this case though.

[yawaramin]

All types are provided in the interface, but admittedly the type for format strings can be a bit difficult to read: Dream_html (dream-html.Dream_html)

I’ll add a note about that.

[octachron]

@R_Huxton

since at the level of understanding I have, in the first one fmt = "%s" c.notes and in the second one fmt = ("%s" c.notes). If this was lisp, I’d understand but I thought avar = x and avar = (x) would be the same.

Just to point out, since I am not sure if you are still confused by this format issue, that fmt = "%s" and not "%s" c.notes. In other words, in

textarea [ id "notescontrol"; name "notes" ] "%s" c.notes;

c.notes is the third argument of the textarea function. So when you write

textarea [ id "notescontrol"; name "notes" ] ("%s" c.notes);

you are suddenly applying the second argument of the textarea function to its third argument, which cannot go anywhere because "%s" is not a function.

I have the impression that the source of your troubles might be that textarea is a “variadic” function like printf whereas the format string "%s %s" is a “normal” GADT with a funky syntax and not something like a macro. For instance writing

textarea [ ] "%s %s" "first" "second"

works.

[R_Huxton]

@yawaramin

All types are provided in the interface, but admittedly the type for format strings can be a bit difficult to read: Dream_html (dream-html.Dream_html)

Aha! I’ll tell you what I didn’t do - I didn’t look in the .mli file at all. Never occurred to me. I haven’t really worked with “header files” since my C days in the 90s. With the code I’ve written I’ve more-or-less just been copying stuff out of the .ml file and not touching it again. Plus of course vscode is probalby plucking bits out of them meaning I don’t need to look. THAT is a good tip for beginners though “make sure to read the .mli” file. Thank you.

Where do you link to that docs page from though? It is saying “No Docs” on dream-html 1.2.0 (latest) · OCaml Package and I don’t see it in your README.md on github either.

[yawaramin]

No problem. I think the OCaml website is having some trouble updating the docs properly. I link to the docs I publish in my repo’s ‘About’ section: GitHub - yawaramin/dream-html: Generate HTML markup from your OCaml Dream backend server

[R_Huxton]

@octachron

@R_Huxton

since at the level of understanding I have, in the first one fmt = "%s" c.notes and in the second one fmt = ("%s" c.notes). If this was lisp, I’d understand but I thought avar = x and avar = (x) would be the same.

Just to point out, since I am not sure if you are still confused by this format issue, that fmt = "%s" and not "%s" c.notes. In other words, in

textarea [ id "notescontrol"; name "notes" ] "%s" c.notes;

c.notes is the third argument of the textarea function. So when you write

OK - another really useful bit of info. No, I did NOT realise this at all. Thank you. The idea that part of the public interface of a library is a partially-applied function is… unexpected and a little unsettling.

This is a case perhaps where the ML lightness of syntax can catch you out until you are familiar with it. If in some more C-inspired language you did this it would end up looking something like:

( textarea([id("notescontrol"), name("notes")], "%s") )(c.notes)

which although clunky makes it quite clear what is going on.

I can imagine though that a type-signature for a variable number of parameters of varying types would be complex.

OK - I think things are cilcking into place.

Two things that I think would definitely be handy in a “how to trouble-shoot” guide:

1. Remember to check the .mli
2. Remember that any function might be partially applied