Re: Re[2]: boolean operators

[David Jones <drj@...>]

On 29 Sep 2006, at 19:24, Gunnar Zötl wrote:

> Nick Gammon wrote:
>
> NG> The lexer can be told whether or not to consider a sign as a  
> separate
> NG> token, I have written ones that do that.
>
> NG> For example, after getting a symbol (x in your example), it is not
> NG> syntactically correct for it to be followed by a number (eg. x  
> 5) so
> NG> the next "+" must be an operator token, not the start of a number.
>
> actually, the unit recognizing such things would be the parser. The
> lexer just knows how to recognize tokens. However, if it is the parser
> recognizing negative numbers, and not the lexer, then the parser can
> easily be written in a way to recognize such stuff.

Actually I think NG was suggesting an approach where the parser,  
having eaten an 'x' say, changes the lexer state so that the next '+'  
is tokenised as part of a number, not as the token '+'.  All the  
compiler writing manuals tell you not to do this, but in real  
compilers it's pretty common.  Isn't that what <FOO> is for in lex?

drj