I am relatively new to Lua language.
As per the book written by R. Ierusalimschy, allwords() function generates words which contain only letters and numbers. I'd like to modify the function so that words can be anything printable between spaces. The mod is shown below, rather more complicated. Is there a way to simplify the code?
The mod would be more handy for Markov Chains Program which is shown in the later chapter.
function allwords () -- the factory
local line = io.read() -- current line
local newline = true
local pos -- word position
return function () -- iterator function
while line do -- repeat while there are lines
local s, e, word
if newline then
line = " "..line.." "
s,e = string.find(line, "%s+", 1)
pos = e + 1 -- advance to the first word
newline = false
s,e = string.find(line, "%s+", pos)
if s then
word = string.sub(line, pos, s-1)
pos = e + 1 -- advance to the next word
line = io.read() -- word not found; try next line
newline = true
return nil -- no more lines: end of traversal