This directory contains some examples illustrating techniques for extracting
high-performance from flex scanners. Each program implements a simplified
version of the Unix "wc" tool: read text from stdin and print the number of
characters, words, and lines present in the text. All programs were compiled
using gcc (version unavailable, sorry) with the -O flag, and run on a
SPARCstation 1+. The input used was a PostScript file, mainly containing
figures, with the following "wc" counts:
lines words characters
214217 635954 2592172
The basic principles illustrated by these programs are:
- match as much text with each rule as possible
- adding rules does not slow you down!
- avoid backing up
and the big caveat that comes with them is:
- you buy performance with decreased maintainability; make
sure you really need it before applying the above techniques.
See the "Performance Considerations" section of flexdoc for more
details regarding these principles.
The different versions of "wc":
mywc.c
a simple but fairly efficient C version
wc1.l a naive flex "wc" implementation
wc2.l somewhat faster; adds rules to match multiple tokens at once
wc3.l faster still; adds more rules to match longer runs of tokens
wc4.l fastest; still more rules added; hard to do much better
using flex (or, I suspect, hand-coding)
wc5.l identical to wc3.l except one rule has been slightly
shortened, introducing backing-up
Timing results (all times in user CPU seconds):
program time notes
------- ---- -----
wc1 16.4 default flex table compression (= -Cem)
wc1 6.7 -Cf compression option
/bin/wc 5.8 Sun's standard "wc" tool
mywc 4.6 simple but better C implementation!
wc2 4.6 as good as C implementation; built using -Cf
wc3 3.8 -Cf
wc4 3.3 -Cf
wc5 5.7 -Cf; ouch, backing up is expensive
84d9c625bf