Tokenise

[Markus Huber <pulse@...>]

I don't know if I should use the Lua language group for this kind of
questions. What I would like to see is a place to discuss general
programming topics in Lua code examples. Especially with Lua fans
because I think Lua fans prefer good style and design. Do you feel
lua-users wiki is a better place for that?


   Token=Tokenise('This.is.a.string','.')

   while Token.Forward do
      print(Token:Forward()) --> This
   end                       --> is
                             --> a
                             --> string

   while Token.Backward do
      print(Token:Backward()) --> string
   end                        --> a
                              --> is
                              --> This

Token.Forward and Token.Backward are pointers to the string. After
reading the last token the correspondig variable is nil.

No question until here. But which output do you expect from the
following code? Note the '\n'

   Token=Tokenise('This.is.a\nstring','\n')
   Token:Forward()                           --> This.is.a
   print(Token:Backward('.'))

1. string

This would be the simple way. Forward and Backward are completly
independent from each other.

2. a

This would be very nice but not so simple to realize. First of all here
Forward and Backward need to hold their own pattern: Forward '.' and
Backward '/n' because if not then reading backward results in 'a\n' On
the other side the tokenise pattern can be much more complicated and
possible in some situations you like to include the forward matching
string when reading the backward? Seems to be not so practical for daily
use. If we get 'a' then it must be only one pointer to mix it / move the
pointer. But what is the first position of this pointer after calling
Tokenise? Hmmm to many questions. You suggestions and wishes are
welcome.


Markus