[svlug] Lex question about variable length fields
Robert Hajime Lanning
lanning at lanning.cc
Thu May 22 16:32:23 PDT 2014
On 05/22/14 15:33, Bob Smith wrote:
> I would like to get three single word tokens followed by a fourth
> token that is whatever is left on the input line. So, the input
> line "My name is Jabba the Hutt" would return tokens
> "is", and
> "Jabba the Hutt".
> The last token looks like several words and when I write the lex
> rules to pick up more than one word, it picks up the whole line.
I think most parsers tokenize the whole line, then piece back together.
This is why, without quoting, multiple spaces are condensed into one.
$ echo this has a lot of spaces
this has a lot of spaces
King of the Potato People
More information about the svlug