What is the eof token in OCaml/OCamlex?

Learning
1

I had:

{
  (*
  #use "lex1.ml";;
  *)
  type result = Int of int | Float of float |
  String of string
}
(* regexps *)
let digit = ['0'-'9'] (* a single digit btw 0 to 9 *)
let digits = digit+ (* at least 1 digit i.e. regex digit digit* *)
let lower_case = ['a'-'z']
let upper_case = ['A'-'Z']
let letter = upper_case | lower_case
let letters = letter+
(*
Rules, what to do when we read the longest char seq from the lexbuffers
note: lexbuf is the implied argument
*)
rule main = parse
  | (digits) '.' digits as f { Float(float_of_string f)::main lexbuf }
  | digits as n { Int (int_of_string n)::main lexbuf }
  | letters as s { String s::main lexbuf}
  | eof {[]}
  | _ {main lexbuf }
{
  let newlexbuf = (Lexing.from_channel stdin) in
    print_newline ();
    main newlexbuf
}

but when I went to ocaml top it never stops. I tried a few things but it doesn’t stop:

 │val main : Lexing.lexbuf -> result list = <fun>
                                                                            │val __ocaml_lex_main_rec : Lexing.lexbuf -> int -> result list = <fun>
                                                                            │
                                                                            │hit there \n
                                                                            │\A
                                                                            │eof
                                                                            │\\n
                                                                            │\\a
                                                                            │\\A
                                                                            │EO

how do I have it know Im done?

1 Like
2

When I try your code above I have no trouble getting it to stop. So I think the problem is that you don’t know how to generate EOF interactively.

I’m using the classic toplevel, which is just named “ocaml”. Here’s the session that shows things working for me:

$ ocamllex m2.mll
7 states, 412 transitions, table size 1690 bytes
$ ocaml
        OCaml version 4.06.1

# #use "m2.ml";;
type result = Int of int | Float of float | String of string
val __ocaml_lex_tables : Lexing.lex_tables =
 ...
val main : Lexing.lexbuf -> result list = <fun>
val __ocaml_lex_main_rec : Lexing.lexbuf -> int -> result list = <fun>

this is a test
^D
- : result list = [String "this"; String "is"; String "a"; String "test"]
#

~

On my system (MacOS) the way to generate EOF from the terminal is to type ^D (hold the control key and type d). The way to do this is different for different systems, but ^D works for systems derived from Unix. It also might be different for fancier toplevels.