From: Stefan Seefeld (seefeld_at_[hidden])
Date: 2005-12-12 10:49:33
Hartmut Kaiser wrote:
> Yes, I agree. Wave should be rewritten (and hopefully will be rewritten) in
> a layered way cleanly implementing every of the mandated translation phases
> each on top of the previous. But I'm inclined to expose iterator interfaces,
> not generators - to allow for rewinds, error handling etc.
Yes !!
Here is another use case worth considering (while we are at it ;-) ):
Imagine a C++ parser interfacing with such a backtracking lexer. It
often needs to parse tentatively, rolling back to a previous state if
the tentative was not successfull.
However, some parts of the tentative may have been successfull nevertheless,
such as the parsing of a nested-name-specifier. It would be very nice if
the parser could somehow mark up such a token sequence by injecting
synthetic tokens back into the lexer so it doesn't need to parse them
again.
A similar use case is described in the 'Decorating tokens...' paper
(http://www.cs.clemson.edu/~malloy/papers/papers.html) where the parsing
requires some (non strictly layered) interaction between parser, symbol
lookup, and lexer.
Just some food for thought...
Regards,
Stefan