Type Lexbuf explanation

Learning
1 
type lexbuf =
  { refill_buff : lexbuf -> unit;
    mutable lex_buffer : bytes;
    mutable lex_buffer_len : int;
    mutable lex_abs_pos : int;
    mutable lex_start_pos : int;
    mutable lex_curr_pos : int;
    mutable lex_last_pos : int;
    mutable lex_last_action : int;
    mutable lex_eof_reached : bool;
    mutable lex_mem : int array;
    mutable lex_start_p : position;
    mutable lex_curr_p : position;
  }

lex_abs_pos is an integer indicating the number of characters read so far?! Or something else? What do lex_start_pos, let_curr_pos, and lex_last_pos mean?

Don’t we already have lex_start_p for the start of the current token, and lex_curr_p for the end of the current token?

2

It’s been forever since the last time I dug thru the meaning of all these various _pos and _p fields, but
(1) remember that some of them are relative to the beginning of the buffer, hence not absolute. E.g. lex_start_pos, lex_curr_pos, but maybe others
(2) lex_abs_pos is absolute
(3) I think the _p fields are there for lexeme_start and lexeme_end and not for the internal use of the lexing algorithms
Whenever I need to understand the meanings of the fields, a quick search around in stdlib/lexing.ml usually does the trick.

2 Likes