1*f4a2713aSLionel Sambuc(*===----------------------------------------------------------------------=== 2*f4a2713aSLionel Sambuc * Lexer Tokens 3*f4a2713aSLionel Sambuc *===----------------------------------------------------------------------===*) 4*f4a2713aSLionel Sambuc 5*f4a2713aSLionel Sambuc(* The lexer returns these 'Kwd' if it is an unknown character, otherwise one of 6*f4a2713aSLionel Sambuc * these others for known things. *) 7*f4a2713aSLionel Sambuctype token = 8*f4a2713aSLionel Sambuc (* commands *) 9*f4a2713aSLionel Sambuc | Def | Extern 10*f4a2713aSLionel Sambuc 11*f4a2713aSLionel Sambuc (* primary *) 12*f4a2713aSLionel Sambuc | Ident of string | Number of float 13*f4a2713aSLionel Sambuc 14*f4a2713aSLionel Sambuc (* unknown *) 15*f4a2713aSLionel Sambuc | Kwd of char 16