From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2005-11-23 11:45:33
Paul Mensonides rote:
> > It would be ok if Wave tool printed this warning and preprocessing
> > continued outputting two tokens '11' and 'LL'.
> > Instead of this it throws wave::cpplexer::lexing_exception
> and stops
> > preprocessing as if '11LL' is invalid character sequence.
> > '11LL' is in no way different from '11ABC' and the latter
> produces two
> > tokens as expected.
>
> Both '11LL' and '11ABC' are valid _preprocessing tokens_.
> However, with long long support, the former is a valid
> _token_ but the latter is not. At the point where
> preprocessing tokens are converted into tokens, an error
> should occur, not two tokens. That point is directly between
> where preprocessing ends and the rest of the phases of
> translation. Normally, a standalone preprocessor doesn't go
> all the way in the conversion phase, but Wave does because it
> is producing tokens by going into phase 7. IMO, there are
> only two options here: 1) Wave should yield preprocessing
> tokens instead of tokens, or 2) Wave should yield an error
> trying to translate '11ABC' into a token (or '11LL' if long
> long is not explicitly enabled--it isn't part of standard C++ yet).
Since Wave produces tokens and not preprocessing tokens I believe the
current behaviour is the correct one.
> > This kind of behavior leads to some problems. For example,
> you can't
> > use Boost.Wave for preprocessing many of the boost files without
> > enabling support for long long suffixes because
> boost/limit.hpp uses
> > them (but they are ifdef'ed away if you define some macroses).
>
> Personally, I think that Wave should produce preprocessing
> tokens. An iterator adaptor could be applied to produce
> tokens (or more accurately, do the checking that tokens require).
Will have to think about this. One of the goals of the Wave library was not
only to make available a preprocessor but also lexer components producing
C++ tokens usable in other contexts as well.
Wave currently doesn't follow strictly the mandated compilation phases. This
has to be changed in the future anyway.
Regards Hartmut