Lines Matching refs:flex
1 This is flex.info, produced by makeinfo version 6.1 from flex.texi.
3 The flex manual is placed under the same licensing conditions as the
4 rest of flex:
40 * flex: (flex). Fast lexical analyzer generator (lex replacement).
44 File: flex.info, Node: Top, Next: Copyright, Prev: (dir), Up: (dir)
46 flex
49 This manual describes 'flex', a tool for generating programs that
53 This edition of 'The flex Manual' documents 'flex' version 2.6.4. It
139 * When was flex born?::
141 * Why do flex scanners call fileno if it is not ANSI compatible?::
142 * Does flex support recursive pattern definitions?::
143 * How do I skip huge chunks of input (tens of megabytes) while using flex?::
152 * If I have a simple grammar cant I just parse it with flex?::
156 * Can I get the flex manual in another format?::
158 * How does flex compile the DFA so quickly?::
171 * Is there a way to make flex treat NULL like a regular character?::
172 * Whenever flex can not match the input it says "flex scanner jammed".::
173 * Why doesn't flex have non-greedy operators like perl does?::
183 * Can you discuss some flex internals?::
186 * Why can't flex understand this variable trailing context pattern?::
189 * Is flex GNU or not?::
195 * Is there a repository for flex scanners?::
196 * How can I conditionally compile or preprocess my flex input file?::
260 File: flex.info, Node: Copyright, Next: Reporting Bugs, Prev: Top, Up: Top
265 The flex manual is placed under the same licensing conditions as the
266 rest of flex:
302 File: flex.info, Node: Reporting Bugs, Next: Introduction, Prev: Copyright, Up: Top
307 If you find a bug in 'flex', please report it using GitHub's issue
308 tracking facility at <https://github.com/westes/flex/issues/>
311 File: flex.info, Node: Introduction, Next: Simple Examples, Prev: Reporting Bugs, Up: Top
316 'flex' is a tool for generating "scanners". A scanner is a program
317 which recognizes lexical patterns in text. The 'flex' program reads the
320 of pairs of regular expressions and C code, called "rules". 'flex'
323 the flex runtime library to produce an executable. When the executable
329 File: flex.info, Node: Simple Examples, Next: Format, Prev: Introduction, Up: Top
334 First some simple examples to get the flavor of how one uses 'flex'.
336 The following 'flex' input specifies a scanner which, when it
343 By default, any text not matched by a 'flex' scanner is copied to the
434 File: flex.info, Node: Format, Next: Patterns, Prev: Simple Examples, Up: Top
439 The 'flex' input file consists of three sections, separated by a line
456 File: flex.info, Node: Definitions Section, Next: Rules Section, Prev: Format, Up: Format
501 before any flex definitions (1). The '%top' block is useful when you
519 File: flex.info, Node: Rules Section, Next: User Code Section, Prev: Definitions Section, Up: F…
524 The "rules" section of the 'flex' input contains a series of rules of
547 File: flex.info, Node: User Code Section, Next: Comments in the Input, Prev: Rules Section, Up:…
558 File: flex.info, Node: Comments in the Input, Prev: User Code Section, Up: Format
564 is considered a comment. Whenever flex encounters a comment, it copies
568 * Comments may not appear in the Rules Section wherever flex is
604 File: flex.info, Node: Patterns, Next: Matching, Prev: Format, Up: Top
727 combinations of 'r/s' that flex cannot match correctly. *Note
738 Note that 'flex''s notion of "newline" is exactly whatever the C
739 compiler used to compile 'flex' interprets '\n' as; in particular,
804 'isblank()', so flex defines '[:blank:]' as a blank or a tab.
814 seen in the 'flex' input. This means the character classes are
815 sensitive to the locale in which 'flex' is executed, and the resulting
827 flex will assume that you meant the literal numeric range, and will
860 but this may change without notice in future revisions of flex.
910 File: flex.info, Node: Matching, Next: Actions, Prev: Patterns, Up: Top
920 the same length, the rule listed first in the 'flex' input file is
931 output. Thus, the simplest valid 'flex' input is:
940 definition 'flex' uses by including one of the special directives
941 '%pointer' or '%array' in the first (definitions) section of your flex
965 section of your 'flex' input. As mentioned above, with '%pointer'
978 File: flex.info, Node: Actions, Next: Generated Scanner, Prev: Matching, Up: Top
1004 balancing '}' is found, and the action may cross multiple lines. 'flex'
1042 text as the originally chosen rule but came later in the 'flex'
1117 Note that 'yyless()' is a macro and can only be used in the flex
1196 File: flex.info, Node: Generated Scanner, Next: Start Conditions, Prev: Actions, Up: Top
1201 The output of 'flex' is the file 'lex.yy.c', which contains the scanning
1222 'flex' generates 'C99' function definitions by default. Flex used to
1229 versions of 'flex' generate standard C99 code only, leaving K&R-style
1245 with previous versions of 'flex', and because it can be used to switch
1301 File: flex.info, Node: Start Conditions, Next: Multiple Input Buffers, Prev: Generated Scanner, …
1306 'flex' provides a mechanism for conditionally activating rules. Any
1335 of the other rules in the 'flex' input. Because of this, exclusive
1608 File: flex.info, Node: Multiple Input Buffers, Next: EOF, Prev: Start Conditions, Up: Top
1614 reading from several input streams. As 'flex' scanners do a large
1622 To negotiate these sorts of problems, 'flex' provides a mechanism for
1663 by flex and will grow as required. This function is intended to be used
1727 letting flex do it).
1820 File: flex.info, Node: EOF, Next: Misc Macros, Prev: Multiple Input Buffers, Up: Top
1831 'flex', after doing the assignment you had to call the special
1869 File: flex.info, Node: Misc Macros, Next: User Values, Prev: EOF, Up: Top
1927 File: flex.info, Node: User Values, Next: Yacc, Prev: Misc Macros, Up: Top
1956 is the file which by default 'flex' reads from. It may be
1959 scanning will have unexpected results since 'flex' buffers its
1984 File: flex.info, Node: Yacc, Next: Scanner Options, Prev: User Values, Up: Top
1989 One of the main uses of 'flex' is as a companion to the 'yacc'
1993 value in the global 'yylval'. To use 'flex' with 'yacc', one specifies
1996 input. This file is then included in the 'flex' scanner. For example,
2009 File: flex.info, Node: Scanner Options, Next: Performance, Prev: Yacc, Up: Top
2014 The various 'flex' options are categorized by function in the following
2037 flex to track line numbers. The last line tells flex what to name the
2041 'flex' also provides a mechanism for controlling options within the
2042 scanner specification itself, rather than from the flex command-line.
2046 of your flex input file.
2053 'flex' scans your rule actions to determine whether you use the
2078 File: flex.info, Node: Options for Specifying Filenames, Next: Options Affecting Scanner Behavior…
2084 instructs flex to write a C header to 'FILE'. This file contains
2099 directs flex to write the scanner to the file 'FILE' instead of
2105 instructs 'flex' to write the scanner it generates to standard
2109 overrides the default skeleton file from which 'flex' constructs
2111 'flex' maintenance or development.
2119 This option is for flex development. We document it here in case
2128 File: flex.info, Node: Options Affecting Scanner Behavior, Next: Code-Level And API Options, Pre…
2134 instructs 'flex' to generate a "case-insensitive" scanner. The
2135 case of letters given in the 'flex' input patterns will be ignored,
2151 instructs 'flex' to generate a "batch" scanner, the opposite of
2160 instructs 'flex' to generate an interactive scanner. An
2171 'flex' scanners default to 'interactive' unless you use the '-Cf'
2174 one of these options, so if you didn't, 'flex' assumes you'd rather
2184 instructs 'flex' to generate a 7-bit scanner, i.e., one which can
2196 which case 'flex' defaults to generating 7-bit scanners unless your
2198 often be the case with non-USA sites). You can tell whether flex
2202 Note that if you use '-Cfe' or '-CFe' 'flex' still defaults to
2208 instructs 'flex' to generate an 8-bit scanner, i.e., one which can
2210 generated using '-Cf' or '-CF', as otherwise flex defaults to
2213 See the discussion of '--7bit' above for 'flex''s default behavior
2220 instructs flex to generate a scanner which always considers its
2228 instructs flex to generate a scanner which never considers its
2233 definition of 'lex'. Since 'flex' was originally designed to
2236 differences between 'flex' and the POSIX standard are:
2243 'abbb'). By default, 'flex' places the precedence of the
2246 '--posix' or '-l' are specified, 'flex' will use the
2265 directs 'flex' to generate a scanner that maintains the number of
2270 modified by 'flex' unless '%option yylineno' is enabled.
2279 File: flex.info, Node: Code-Level And API Options, Next: Options for Scanner Speed and Size, Pre…
2291 instructs flex to generate a C scanner that is meant to be called
2298 instruct flex that 'GNU bison' '%locations' are being used. This
2303 instructs 'flex' not to generate '#line' directives. Without this
2304 option, 'flex' peppers the generated scanner with '#line'
2306 located with respect to either the original 'flex' input file (if
2308 the errors are 'flex''s fault - you should report these sorts of
2312 instructs flex to generate a reentrant C scanner. The generated
2316 reentrant and non-reentrant 'flex' scanners, non-reentrant flex
2324 specifies that you want flex to generate a C++ scanner class.
2335 changes the default 'yy' prefix used by 'flex' for all
2366 This option lets you easily link together multiple 'flex' programs
2374 directs flex to provide a default 'main()' program for the scanner,
2381 exist. Be aware that certain options may cause flex to generate
2389 It informs 'flex' that you have derived 'NAME' as a subclass of
2390 'yyFlexLexer', so 'flex' will place your actions in the member
2397 File: flex.info, Node: Options for Scanner Speed and Size, Next: Debugging Options, Prev: Code-L…
2412 ("align") instructs flex to trade off larger tables in the
2421 directs 'flex' to construct "equivalence classes", i.e., sets
2423 example, if the only appearance of digits in the 'flex' input
2433 'flex' should not compress the tables by taking advantages of
2442 directs 'flex' to construct "meta-equivalence classes", which
2468 The default setting is '-Cem', which specifies that 'flex' should
2519 File: flex.info, Node: Debugging Options, Next: Miscellaneous Options, Prev: Options for Scanner…
2543 defining the scanner (i.e., the file that was fed to flex).
2551 comments regarding features of the 'flex' input file which will
2568 makes 'flex' run in "trace" mode. It will generate a lot of
2571 This option is mostly for use in maintaining 'flex'.
2577 specifies that 'flex' should write to 'stderr' a summary of
2579 statistics are meaningless to the casual 'flex' user, but the first
2580 line identifies the version of 'flex' (same as reported by
2586 be matched but no default rule has been given, the flex will warn
2590 File: flex.info, Node: Miscellaneous Options, Prev: Debugging Options, Up: Scanner Options
2599 generates a "help" summary of 'flex''s options to 'stdout' and then
2609 File: flex.info, Node: Performance, Next: Cxx, Prev: Scanner Options, Up: Top
2614 The main design goal of 'flex' is that it generate high-performance
2645 not possibly match newlines, since flex does not need to check them for
2738 accidentally match a valid token. A possible future 'flex' feature will
2854 'flex' can't figure this out, and it will plan for possibly needing to
2876 Compiled with '-Cf', this is about as fast as one can get a 'flex'
2879 A final note: 'flex' is slow when matching 'NUL's, particularly when
2892 File: flex.info, Node: Cxx, Next: Reentrant, Prev: Performance, Up: Top
2900 'flex' provides two different ways to generate scanners for use with
2901 C++. The first way is to simply compile a scanner generated by 'flex'
2909 You can also use 'flex' to generate a C++ scanner class, using the
2911 specified if the name of the 'flex' executable ends in a '+', such as
2912 'flex++'. When using this option, 'flex' defaults to generating the
2962 performs the same role is 'yylex()' does for ordinary 'flex'
2967 to inform 'flex' that you will be using that subclass instead of
2969 'yyFlexLexer::yylex()', 'flex' generates 'S::yylex()' (and also
3019 // An example of using the flex C++ scanner class.
3100 File: flex.info, Node: Reentrant, Next: Lex and Posix, Prev: Cxx, Up: Top
3105 'flex' has the ability to generate a reentrant C scanner. This is
3110 reentrant 'flex' scanner without the need for synchronization with other
3122 File: flex.info, Node: Reentrant Uses, Next: Reentrant Overview, Prev: Reentrant, Up: Reentrant
3173 File: flex.info, Node: Reentrant Overview, Next: Reentrant Example, Prev: Reentrant Uses, Up: R…
3192 'flex' variables.
3197 File: flex.info, Node: Reentrant Example, Next: Reentrant Detail, Prev: Reentrant Overview, Up:…
3229 File: flex.info, Node: Reentrant Detail, Next: Reentrant Functions, Prev: Reentrant Example, Up…
3235 'flex'.
3248 File: flex.info, Node: Specify Reentrant, Next: Extra Reentrant Argument, Prev: Reentrant Detail…
3256 (*note Reentrant Example::. Had this option not been specified, 'flex'
3263 File: flex.info, Node: Extra Reentrant Argument, Next: Global Replacement, Prev: Specify Reentra…
3279 both functions. In fact, all 'flex' functions in a reentrant scanner
3289 File: flex.info, Node: Global Replacement, Next: Init and Destroy Functions, Prev: Extra Reentra…
3294 All global variables in traditional flex have been replaced by macro
3299 lvalue. All of the familiar 'flex' globals have been replaced by their
3316 File: flex.info, Node: Init and Destroy Functions, Next: Accessor Methods, Prev: Global Replacem…
3357 a scanner if you plan to reuse it. A 'flex' scanner (both reentrant and
3378 File: flex.info, Node: Accessor Methods, Next: Extra Data, Prev: Init and Destroy Functions, Up…
3383 Accessor methods (get/set functions) provide access to common 'flex'
3387 Portions of your project will need access to 'flex' values, such as
3390 are no global 'flex' values. You can not access them directly.
3391 Instead, you must access 'flex' values using accessor methods (get/set
3393 where 'NAME' is the name of the 'flex' variable you want. For example:
3412 File: flex.info, Node: Extra Data, Next: About yyscan_t, Prev: Accessor Methods, Up: Reentrant …
3474 File: flex.info, Node: About yyscan_t, Prev: Extra Data, Up: Reentrant Detail
3488 File: flex.info, Node: Reentrant Functions, Prev: Reentrant Detail, Up: Reentrant
3525 'flex' unless '%option yylineno' is enabled. This is to allow the user
3526 to maintain the line count independently of 'flex'.
3545 'flex' input.
3548 File: flex.info, Node: Lex and Posix, Next: Memory Management, Prev: Reentrant, Up: Top
3553 'flex' is a rewrite of the AT&T Unix _lex_ tool (the two implementations
3556 write scanners acceptable to both implementations. 'flex' is fully
3560 we discuss all of the known areas of incompatibility between 'flex',
3561 AT&T 'lex', and the POSIX specification. 'flex''s '-l' option turns on
3565 option. 'flex' is fully compatible with 'lex' with the following
3584 * The 'flex' restriction that 'input()' cannot be redefined is in
3592 * 'flex' scanners are not as reentrant as 'lex' scanners. In
3597 fatal flex scanner internal error--end of buffer missed
3605 Reentrant::, for 'flex''s reentrant API.
3607 * Also note that 'flex' C++ scanner classes _are_ reentrant, so if
3619 * When definitions are expanded, 'flex' encloses them in parentheses.
3629 is such that the '?' is associated with '[A-Z0-9]*'. With 'flex',
3636 the '<s>', '/', and '<<EOF>>' operators cannot be used in a 'flex'
3652 'flex' does not support this feature.
3664 match one, two, or three occurrences of 'abc'", whereas 'flex'
3671 'bar' anywhere", whereas 'flex' interprets it as "match either
3676 are not required by 'flex' scanners.. 'flex' ignores them.
3678 for use with either 'flex' or 'lex'. Scanners also include
3680 'YY_FLEX_SUBMINOR_VERSION' indicating which version of 'flex'
3683 'flex' being used is a beta version, then the symbol 'FLEX_BETA' is
3689 The following 'flex' features are not included in 'lex' or the POSIX
3711 * almost all of the 'flex' command-line options
3714 'flex' you can put multiple actions on the same line, separated with
3723 'flex' does not truncate the action. Actions that are not enclosed
3727 File: flex.info, Node: Memory Management, Next: Serialized Tables, Prev: Lex and Posix, Up: Top
3732 This chapter describes how flex handles dynamic memory, and how you can
3742 File: flex.info, Node: The Default Memory Management, Next: Overriding The Default Memory Managem…
3749 first call to yylex(). Thereafter, flex may reallocate more memory if
3759 largest chunk of dynamic memory flex consumes. This buffer will
3796 this rule is that flex will delete the current buffer automatically
3798 be sure to set it to NULL. That way, flex will not try to delete
3800 time of this writing, flex does not provide a growable stack for
3813 to flex.
3816 File: flex.info, Node: Overriding The Default Memory Management, Next: A Note About yytext And Me…
3825 telling flex that you will provide your own implementations.
3892 File: flex.info, Node: A Note About yytext And Memory, Prev: Overriding The Default Memory Manage…
3897 When flex finds a match, 'yytext' points to the first character of the
3908 (commonly used with flex), you will discover that the error recovery
3917 File: flex.info, Node: Serialized Tables, Next: Diagnostics, Prev: Memory Management, Up: Top
3922 A 'flex' scanner has the ability to save the DFA tables to a file, and
3942 File: flex.info, Node: Creating Serialized Tables, Next: Loading and Unloading Serialized Tables,…
3953 These options instruct flex to save the DFA tables to the file FILE.
3963 the serialized tables into one file, and flex will find the correct set
3967 $ flex --tables-file --prefix=cpp cpp.l
3968 $ flex --tables-file --prefix=c c.l
3975 At runtime, we will open the file and tell flex to load the tables from
3980 File: flex.info, Node: Loading and Unloading Serialized Tables, Next: Tables File Format, Prev: …
4016 File: flex.info, Node: Tables File Format, Prev: Loading and Unloading Serialized Tables, Up: Se…
4021 This section defines the file format of serialized 'flex' tables.
4087 This is the version of flex that was used to create the serialized
4187 File: flex.info, Node: Diagnostics, Next: Limitations, Prev: Serialized Tables, Up: Top
4192 The following is a list of 'flex' diagnostic messages:
4214 'yymore()' but that 'flex' failed to notice the fact, meaning that
4215 'flex' scanned the first two sections looking for occurrences of
4218 '%option yymore' to indicate to 'flex' that you really do use these
4221 * 'flex scanner jammed'. a scanner compiled with '-s' has
4228 'YYLMAX' in the definitions section of your 'flex' input.
4237 * 'flex scanner push-back overflow'. you used 'unput()' to push back
4248 * 'fatal flex scanner internal error--end of buffer missed'. This
4260 File: flex.info, Node: Limitations, Next: Bibliography, Prev: Diagnostics, Up: Top
4289 '<stdio.h>' routines, such as, getchar(), with 'flex' rules and expect
4297 The 'flex' internal algorithms need documentation.
4300 File: flex.info, Node: Bibliography, Next: FAQ, Prev: Limitations, Up: Top
4320 pattern-matching techniques used by 'flex' (deterministic finite
4324 File: flex.info, Node: FAQ, Next: Appendices, Prev: Bibliography, Up: Top
4329 From time to time, the 'flex' maintainer receives certain questions.
4335 * When was flex born?::
4337 * Why do flex scanners call fileno if it is not ANSI compatible?::
4338 * Does flex support recursive pattern definitions?::
4339 * How do I skip huge chunks of input (tens of megabytes) while using flex?::
4348 * If I have a simple grammar cant I just parse it with flex?::
4352 * Can I get the flex manual in another format?::
4354 * How does flex compile the DFA so quickly?::
4367 * Is there a way to make flex treat NULL like a regular character?::
4368 * Whenever flex can not match the input it says "flex scanner jammed".::
4369 * Why doesn't flex have non-greedy operators like perl does?::
4379 * Can you discuss some flex internals?::
4382 * Why can't flex understand this variable trailing context pattern?::
4385 * Is flex GNU or not?::
4391 * Is there a repository for flex scanners?::
4392 * How can I conditionally compile or preprocess my flex input file?::
4439 File: flex.info, Node: When was flex born?, Next: How do I expand backslash-escape sequences in C…
4441 When was flex born?
4449 …flex.info, Node: How do I expand backslash-escape sequences in C-style quoted strings?, Next: Wh…
4471 File: flex.info, Node: Why do flex scanners call fileno if it is not ANSI compatible?, Next: Does…
4473 Why do flex scanners call fileno if it is not ANSI compatible?
4485 …flex.info, Node: Does flex support recursive pattern definitions?, Next: How do I skip huge chun…
4487 Does flex support recursive pattern definitions?
4496 power of regular expressions in general (and therefore flex scanners,
4504 …flex.info, Node: How do I skip huge chunks of input (tens of megabytes) while using flex?, Next:…
4506 How do I skip huge chunks of input (tens of megabytes) while using flex?
4512 …flex.info, Node: Flex is not matching my patterns in the same order that I defined them., Next: …
4517 'flex' picks the rule that matches the most text (i.e., the longest
4518 possible input string). This is because 'flex' uses an entirely
4525 matches more than one rule, 'flex' scanners pick the rule that matched
4529 If you want 'flex' to choose a shorter match, then you can work
4546 File: flex.info, Node: My actions are executing out of order or sometimes not at all., Next: How …
4559 'flex' requires that the opening '{' of an action associated with a
4568 File: flex.info, Node: How can I have multiple input sources feed into the same scanner at the sam…
4574 * your scanner is free of backtracking (verified using 'flex''s '-b'
4601 both reads keyboard input using a 'flex' scanner and IPC traffic from
4605 File: flex.info, Node: Can I build nested parsers that work with the same input file?, Next: How …
4611 that 'flex' block-buffers the input it reads from 'yyin'. This means
4621 File: flex.info, Node: How can I match text only at the end of a file?, Next: How can I make REJE…
4636 File: flex.info, Node: How can I make REJECT cascade across start condition boundaries?, Next: Wh…
4663 File: flex.info, Node: Why cant I use fast or full tables with interactive mode?, Next: How much …
4668 One of the assumptions flex makes is that interactive applications are
4685 File: flex.info, Node: How much faster is -F or -f than -C?, Next: If I have a simple grammar can…
4693 File: flex.info, Node: If I have a simple grammar cant I just parse it with flex?, Next: Why does…
4695 If I have a simple grammar can't I just parse it with flex?
4703 …flex.info, Node: Why doesn't yyrestart() set the start state back to INITIAL?, Next: How can I m…
4710 that beginning with 'flex' version 2.4, use of 'yyrestart()' is no
4715 File: flex.info, Node: How can I match C-style comments?, Next: The period isn't working the way …
4745 File: flex.info, Node: The period isn't working the way I expected., Next: Can I get the flex man…
4772 File: flex.info, Node: Can I get the flex manual in another format?, Next: Does there exist a "fa…
4774 Can I get the flex manual in another format?
4777 The 'flex' source distribution includes a texinfo manual. You are free
4782 …ile: flex.info, Node: Does there exist a "faster" NDFA->DFA algorithm?, Next: How does flex comp…
4793 File: flex.info, Node: How does flex compile the DFA so quickly?, Next: How can I use more than 8…
4795 How does flex compile the DFA so quickly?
4798 There are two big speed wins that 'flex' uses:
4812 …flex.info, Node: How can I use more than 8192 rules?, Next: How do I abandon a file in the middl…
4819 'flex' with the following changes in 'flexdef.h':
4847 File: flex.info, Node: How do I abandon a file in the middle of a scan and switch to a new file?, …
4857 File: flex.info, Node: How do I execute code only during initialization (only before the first sca…
4876 File: flex.info, Node: How do I execute code at termination?, Next: Where else can I find help?, …
4884 File: flex.info, Node: Where else can I find help?, Next: Can I include comments in the "rules" s…
4889 You can find the flex homepage on the web at
4890 <http://flex.sourceforge.net/>. See that page for details about flex
4894 File: flex.info, Node: Can I include comments in the "rules" section of the file?, Next: I get an…
4903 File: flex.info, Node: I get an error about undefined yywrap()., Next: How can I change the match…
4916 File: flex.info, Node: How can I change the matching pattern at run time?, Next: How can I expand…
4921 You can't, it's compiled into a static table when flex builds the
4925 File: flex.info, Node: How can I expand macros in the input?, Next: How can I build a two-pass sc…
4960 File: flex.info, Node: How can I build a two-pass scanner?, Next: How do I match any string not m…
4978 File: flex.info, Node: How do I match any string not matched in the preceding rules?, Next: I am …
4985 then the first rule listed in the 'flex' input wins, e.g.,
4994 that case the 'flex' scanner will pick the first rule listed in your
4998 File: flex.info, Node: I am trying to port code from AT&T lex that uses yysptr and yysbuf., Next:…
5008 shouldn't need to (and must not) replace 'flex''s 'unput()' function.
5011 …ile: flex.info, Node: Is there a way to make flex treat NULL like a regular character?, Next: Wh…
5013 Is there a way to make flex treat NULL like a regular character?
5017 ancient version of 'flex'. The latest release is version 2.6.4.
5020 …flex.info, Node: Whenever flex can not match the input it says "flex scanner jammed"., Next: Why…
5022 Whenever flex can not match the input it says "flex scanner jammed".
5036 …flex.info, Node: Why doesn't flex have non-greedy operators like perl does?, Next: Memory leak -…
5038 Why doesn't flex have non-greedy operators like perl does?
5044 to implement than longest leftmost match (which flex does).
5062 File: flex.info, Node: Memory leak - 16386 bytes allocated by malloc., Next: How do I track the b…
5067 UPDATED 2002-07-10: As of 'flex' version 2.5.9, this leak means that you
5069 'flex', then read on.
5074 reentrant scanner, NOT in the C++ scanner). Since 'flex' doesn't know
5092 File: flex.info, Node: How do I track the byte offset for lseek()?, Next: How do I use my own I/O…
5103 it's possible that 'flex' will request less than 'YY_READ_BUF_SIZE'
5106 second problem is that when refilling its internal buffer, 'flex' keeps
5123 File: flex.info, Node: How do I use my own I/O classes in a C++ scanner?, Next: How do I skip as …
5128 When the flex C++ scanning class rewrite finally happens, then this sort
5140 File: flex.info, Node: How do I skip as many chars as possible?, Next: deleteme00, Prev: How do …
5174 File: flex.info, Node: deleteme00, Next: Are certain equivalent patterns faster than others?, Pr…
5180 When was flex born?
5188 File: flex.info, Node: Are certain equivalent patterns faster than others?, Next: Is backing up a…
5199 [Note, the most recent flex release is 2.5.4, which you can get from
5206 > (in a very complicated flex program) caused the program to slow from
5231 variable length, which flex -p will tell you, then this reflects a basic
5265 File: flex.info, Node: Is backing up a big deal?, Next: Can I fake multi-byte character support?,…
5317 File: flex.info, Node: Can I fake multi-byte character support?, Next: deleteme01, Prev: Is back…
5323 Subject: Re: flex - multi-byte support?
5329 > range of expected character code values (in octal format), flex will
5349 have a very old version of flex. You can get the most recent, 2.5.4, from
5355 File: flex.info, Node: deleteme01, Next: Can you discuss some flex internals?, Prev: Can I fake …
5366 Unfortunately flex at the moment has a widespread assumption within it
5374 Fixing flex to handle wider characters is on the long-term to-do list.
5375 But since flex is a strictly spare-time project these days, this probably
5381 File: flex.info, Node: Can you discuss some flex internals?, Next: unput() messes up yy_at_bol, …
5383 Can you discuss some flex internals?
5387 Subject: Re: translation of flex
5393 > working with flex. I have a few questions about some of the messages which
5396 All of the things you're wondering about, by the way, concerning flex
5436 File: flex.info, Node: unput() messes up yy_at_bol, Next: The | operator is not doing what I want…
5458 > By the way I am using flex 2.5.2 and using the "-l".
5466 File: flex.info, Node: The | operator is not doing what I want, Next: Why can't flex understand t…
5499 from your use of '|' later confusing flex.
5506 File: flex.info, Node: Why can't flex understand this variable trailing context pattern?, Next: T…
5508 Why can't flex understand this variable trailing context pattern?
5512 Subject: Re: flex-2.5.3 bug report
5518 > flex to process it. Could you try it and maybe point me in the right direction?
5525 option for AT&T lex compatibility. Without this option, flex automatically
5531 …e: flex.info, Node: The ^ operator isn't working, Next: Trailing context is getting confused wit…
5555 then it won't work, because flex pushes back macro definitions enclosed
5567 File: flex.info, Node: Trailing context is getting confused with trailing optional patterns, Next…
5592 so flex's matching will be correct.
5597 File: flex.info, Node: Is flex GNU or not?, Next: ERASEME53, Prev: Trailing context is getting c…
5599 Is flex GNU or not?
5611 Well, strictly speaking flex isn't part of the GNU project. They just
5630 use flex.
5637 File: flex.info, Node: ERASEME53, Next: I need to scan if-then-else blocks and while loops, Prev…
5662 File: flex.info, Node: I need to scan if-then-else blocks and while loops, Next: ERASEME55, Prev…
5683 have been on the to-do list for a while. As flex is a purely spare-time
5690 File: flex.info, Node: ERASEME55, Next: ERASEME56, Prev: I need to scan if-then-else blocks and …
5719 File: flex.info, Node: ERASEME56, Next: ERASEME57, Prev: ERASEME55, Up: FAQ
5755 File: flex.info, Node: ERASEME57, Next: Is there a repository for flex scanners?, Prev: ERASEME5…
5761 Subject: Re: flex limitations
5771 What version of flex are you using? If I feed this to 2.5.4, it complains:
5779 (With the introduction of start condition scopes, flex can't accommodate
5787 File: flex.info, Node: Is there a repository for flex scanners?, Next: How can I conditionally co…
5789 Is there a repository for flex scanners?
5795 …flex.info, Node: How can I conditionally compile or preprocess my flex input file?, Next: Where …
5797 How can I conditionally compile or preprocess my flex input file?
5804 …flex.info, Node: Where can I find grammars for lex and yacc?, Next: I get an end-of-buffer messa…
5809 In the sources for flex and bison.
5812 File: flex.info, Node: I get an end-of-buffer message for each character scanned., Next: unnamed-…
5826 File: flex.info, Node: unnamed-faq-62, Next: unnamed-faq-63, Prev: I get an end-of-buffer messag…
5837 > I took a quick look into the flex-sources and altered some #defines in
5859 File: flex.info, Node: unnamed-faq-63, Next: unnamed-faq-64, Prev: unnamed-faq-62, Up: FAQ
5891 File: flex.info, Node: unnamed-faq-64, Next: unnamed-faq-65, Prev: unnamed-faq-63, Up: FAQ
5902 > Can you explain to me what is ment by a long-jump in relation to flex?
5906 > what is the flex activation frame.
5911 > file and using flex++ isnot really an option!
5920 File: flex.info, Node: unnamed-faq-65, Next: unnamed-faq-66, Prev: unnamed-faq-64, Up: FAQ
5946 File: flex.info, Node: unnamed-faq-66, Next: unnamed-faq-67, Prev: unnamed-faq-65, Up: FAQ
5967 you are probably running into is that when flex expands a name definition,
5976 File: flex.info, Node: unnamed-faq-67, Next: unnamed-faq-68, Prev: unnamed-faq-66, Up: FAQ
5990 > works well when using LEX instead of flex. Do you have some ideas about the
5995 "extern char *yytext" (which is what flex uses). If it's not that, then
6001 File: flex.info, Node: unnamed-faq-68, Next: unnamed-faq-69, Prev: unnamed-faq-67, Up: FAQ
6007 Subject: Re: flex 2.5: c++ scanners & start conditions
6028 File: flex.info, Node: unnamed-faq-69, Next: unnamed-faq-70, Prev: unnamed-faq-68, Up: FAQ
6034 Subject: Re: current position in flex buffer
6039 > The problem is how to determine the current position in flex active
6045 The latest flex release, by the way, is 2.5.4, available from ftp.ee.lbl.gov.
6050 File: flex.info, Node: unnamed-faq-70, Next: unnamed-faq-71, Prev: unnamed-faq-69, Up: FAQ
6073 File: flex.info, Node: unnamed-faq-71, Next: unnamed-faq-72, Prev: unnamed-faq-70, Up: FAQ
6079 Subject: Re: flex: how to control start condition from parser?
6096 flex development has come to a virtual stand-still :-(, so a workaround
6102 File: flex.info, Node: unnamed-faq-72, Next: unnamed-faq-73, Prev: unnamed-faq-71, Up: FAQ
6108 Subject: Re: freebsd flex bug?
6116 it by using flex -L, so it doesn't generate #line directives in its
6118 name that is also a variable name, or something like that; flex spits
6132 since flex will put "#define foo 1" in the generated scanner.
6137 File: flex.info, Node: unnamed-faq-73, Next: unnamed-faq-74, Prev: unnamed-faq-72, Up: FAQ
6143 Subject: Re: Lost flex .l file
6163 File: flex.info, Node: unnamed-faq-74, Next: unnamed-faq-75, Prev: unnamed-faq-73, Up: FAQ
6181 A final note: flex is slow when matching NUL's, particularly
6191 File: flex.info, Node: unnamed-faq-75, Next: unnamed-faq-76, Prev: unnamed-faq-74, Up: FAQ
6229 File: flex.info, Node: unnamed-faq-76, Next: unnamed-faq-77, Prev: unnamed-faq-75, Up: FAQ
6250 File: flex.info, Node: unnamed-faq-77, Next: unnamed-faq-78, Prev: unnamed-faq-76, Up: FAQ
6256 Subject: Re: flex problem
6273 File: flex.info, Node: unnamed-faq-78, Next: unnamed-faq-79, Prev: unnamed-faq-77, Up: FAQ
6305 File: flex.info, Node: unnamed-faq-79, Next: unnamed-faq-80, Prev: unnamed-faq-78, Up: FAQ
6338 > flex 2.5.4 binary works.
6352 File: flex.info, Node: unnamed-faq-80, Next: unnamed-faq-81, Prev: unnamed-faq-79, Up: FAQ
6358 Subject: Re: flex output for static code portion
6363 > I would like to use flex under the hood to generate a binary file
6375 > I was wondering if you know of anyone who has used flex in this way.
6378 numerous other flex tweaks :-).
6383 File: flex.info, Node: unnamed-faq-81, Next: unnamed-faq-82, Prev: unnamed-faq-80, Up: FAQ
6397 Subject: "flex scanner push-back overflow"
6411 to include some lengthy lists into a .l file. Following is a flex macro
6416 The complete list contains about 10kB. When I try to "flex" this file
6417 (on a Solaris 2.6 machine, using a modified flex 2.5.4 (I only increased
6420 myflex/flex -8 sentag.tmp.l
6421 flex scanner push-back overflow
6438 File: flex.info, Node: unnamed-faq-82, Next: unnamed-faq-83, Prev: unnamed-faq-81, Up: FAQ
6444 Subject: Re: "flex scanner push-back overflow"
6449 > myflex/flex -8 sentag.tmp.l
6450 > flex scanner push-back overflow
6452 Flex itself uses a flex scanner. That scanner is running out of buffer
6458 perhaps there's another/better way to do it; or to rebuild flex's own
6466 File: flex.info, Node: unnamed-faq-83, Next: unnamed-faq-84, Prev: unnamed-faq-82, Up: FAQ
6512 File: flex.info, Node: unnamed-faq-84, Next: unnamed-faq-85, Prev: unnamed-faq-83, Up: FAQ
6518 Subject: Re: Problems with restarting flex-2.5.2-generated scanner
6523 > I am using flex-2.5.2 and bison 1.25 for Solaris and I am desperately
6538 File: flex.info, Node: unnamed-faq-85, Next: unnamed-faq-86, Prev: unnamed-faq-84, Up: FAQ
6544 Subject: Re: flex 2.5.4a
6571 File: flex.info, Node: unnamed-faq-86, Next: unnamed-faq-87, Prev: unnamed-faq-85, Up: FAQ
6603 File: flex.info, Node: unnamed-faq-87, Next: unnamed-faq-88, Prev: unnamed-faq-86, Up: FAQ
6609 Subject: Re: flex input buffer
6614 > In the flex.skl file the size of the default input buffers is set. Can you
6623 File: flex.info, Node: unnamed-faq-88, Next: unnamed-faq-90, Prev: unnamed-faq-87, Up: FAQ
6636 > "flex: input rules are too complicated (>= 32000 NFA states)"
6650 File: flex.info, Node: unnamed-faq-90, Next: unnamed-faq-91, Prev: unnamed-faq-88, Up: FAQ
6685 File: flex.info, Node: unnamed-faq-91, Next: unnamed-faq-92, Prev: unnamed-faq-90, Up: FAQ
6703 Subject: A question on flex C++ scanner
6711 I have been using flex for years.
6715 on C++ lanuage. Thanks to your enhancement, flex did
6736 as I checked the source. (flex 2.5.4)
6753 File: flex.info, Node: unnamed-faq-92, Next: unnamed-faq-93, Prev: unnamed-faq-91, Up: FAQ
6759 Subject: Re: A question on flex C++ scanner
6772 File: flex.info, Node: unnamed-faq-93, Next: unnamed-faq-94, Prev: unnamed-faq-92, Up: FAQ
6796 File: flex.info, Node: unnamed-faq-94, Next: unnamed-faq-95, Prev: unnamed-faq-93, Up: FAQ
6802 Subject: Re: flex - question
6832 File: flex.info, Node: unnamed-faq-95, Next: unnamed-faq-96, Prev: unnamed-faq-94, Up: FAQ
6845 > I tried compiling (gnu)flex on a Solaris 2.4 machine
6850 > ./flex -t -p ./scan.l >scan.c
6851 > sh: ./flex: not found
6857 > trying to install flex now. I then edited the Makefile to
6858 > and changed where it says "FLEX = flex" to "FLEX = lex"
6860 > the "-p" option. Is there any way I can compile flex without
6861 > using flex or lex?
6866 initscan.c to scan.c in order to build flex. Try fetching a fresh
6873 File: flex.info, Node: unnamed-faq-96, Next: unnamed-faq-97, Prev: unnamed-faq-95, Up: FAQ
6894 If this last tries to first build scan.c from scan.l using ./flex, then
6900 File: flex.info, Node: unnamed-faq-97, Next: unnamed-faq-98, Prev: unnamed-faq-96, Up: FAQ
6920 File: flex.info, Node: unnamed-faq-98, Next: unnamed-faq-99, Prev: unnamed-faq-97, Up: FAQ
6938 For flex, you can probably do the equivalent using a switch on YYSTATE.
6943 File: flex.info, Node: unnamed-faq-99, Next: unnamed-faq-100, Prev: unnamed-faq-98, Up: FAQ
6949 Subject: Re: Regarding distribution of flex and yacc based grammars
6956 > for the generated C files from flex and bison since they are generated by
6957 > flex and bison ?
6959 For flex, no. I don't know what the current state of this is for bison.
6962 > the grammar files which are fed into flex and bison ?
6964 Again, for flex, no.
6966 See the file "COPYING" in the flex distribution for the legalese.
6971 File: flex.info, Node: unnamed-faq-100, Next: unnamed-faq-101, Prev: unnamed-faq-99, Up: FAQ
6987 You can't do this - flex is *not* a parser like yacc (which does indeed
6993 File: flex.info, Node: unnamed-faq-101, Next: What is the difference between YYLEX_PARAM and YY_D…
7023 File: flex.info, Node: What is the difference between YYLEX_PARAM and YY_DECL?, Next: Why do I ge…
7028 YYLEX_PARAM is not a flex symbol. It is for Bison. It tells Bison to
7037 File: flex.info, Node: Why do I get "conflicting types for yylex" error?, Next: How do I access t…
7047 File: flex.info, Node: How do I access the values set in a Flex action from within a Bison action?…
7056 File: flex.info, Node: Appendices, Next: Indices, Prev: FAQ, Up: Top
7069 File: flex.info, Node: Makefiles and Flex, Next: Bison Bridge, Prev: Appendices, Up: Appendices
7079 'flex', however, the '.l' files are the sources, and the generated '.c'
7092 LEX=flex
7104 LEX=flex
7125 Finally, we provide a realistic example of a 'flex' scanner used with
7127 Since a 'flex' scanner will typically include a header file (e.g.,
7135 LEX = flex
7161 implicit rules for flex-generated scanners.
7163 (2) GNU 'automake' may generate code to execute flex in
7170 File: flex.info, Node: Bison Bridge, Next: M4 Dependency, Prev: Makefiles and Flex, Up: Appendi…
7175 This section describes the 'flex' features useful when integrating
7176 'flex' with 'GNU bison'(1). Skip this section if you are not using
7177 'bison' with your scanner. Here we discuss only the 'flex' half of the
7178 'flex' and 'bison' pair. We do not discuss 'bison' in any detail. For
7183 bison-bridge' or by supplying '--bison-bridge' when invoking 'flex' from
7184 the command line. This instructs 'flex' that the macro 'yylval' may be
7186 header file, included in section 1 of the 'flex' input file. For a list
7199 Support for 'yylloc' is optional in 'bison', so it is optional in 'flex'
7200 as well. The following is an example of a 'flex' scanner that is
7246 means the only way to use flex with bison. We merely provide some glue
7250 File: flex.info, Node: M4 Dependency, Next: Common Patterns, Prev: Bison Bridge, Up: Appendices
7255 The macro processor 'm4'(1) must be installed wherever flex is
7256 installed. 'flex' invokes 'm4', found by searching the directories in
7275 'm4' is only required at the time you run 'flex'. The generated
7280 (1) The use of m4 is subject to change in future revisions of flex.
7281 It is not part of the public API of flex. Do not depend on it.
7284 File: flex.info, Node: Common Patterns, Prev: M4 Dependency, Up: Appendices
7300 File: flex.info, Node: Numbers, Next: Identifiers, Up: Common Patterns
7335 File: flex.info, Node: Identifiers, Next: Quoted Constructs, Prev: Numbers, Up: Common Patterns
7354 File: flex.info, Node: Quoted Constructs, Next: Addresses, Prev: Identifiers, Up: Common Patter…
7386 File: flex.info, Node: Addresses, Prev: Quoted Constructs, Up: Common Patterns
7421 File: flex.info, Node: Indices, Prev: Appendices, Up: Top
7436 File: flex.info, Node: Concept Index, Next: Index of Functions and Macros, Prev: Indices, Up: I…
7474 * bison, bridging with flex: Bison Bridge. (line 6)
7482 * C code in flex input: Definitions Section. (line 40)
7499 * comments in flex input: Definitions Section. (line 37)
7513 * copyright of flex: Copyright. (line 6)
7520 * Definitions, in flex input: Definitions Section. (line 6)
7523 * distributing flex: Copyright. (line 6)
7526 * embedding C code in flex input: Definitions Section. (line 40)
7552 * format of flex input: Format. (line 6)
7578 * limitations of flex: Limitations. (line 6)
7609 * non-POSIX features of flex: Lex and Posix. (line 142)
7666 * reentrant, accessing flex variables: Global Replacement. (line 6)
7684 * rules, in flex input: Rules Section. (line 6)
7686 * sections of flex input: Format. (line 6)
7738 * user code, in flex input: User Code Section. (line 6)