Re: Token filter

[Thomas Lauer <thomas.lauer@...>]

I have further investigated this and found out the following. The
original token filter file looks like that:

function FILTER(get,source)
	print("FILTER",source)
	FILTER=function ()
		local line,token,value=get()
		print("FILTER",line,token,value)
		return line,token,value
	end
end

My problem comes from the fact that the filter function replaces itself.
Or, in other words, that one function does two things.

Whenever a new file is loaded, luaX_setinput() (the proxy.c version)
installs a filter like that:

void luaX_setinput(lua_State *L, LexState *ls, ZIO *z, TString *source)
{
	 setinput(L,ls,z,source);
	 lua_getglobal(L,FILTER);
	 if (lua_isnil(L,-1)) lua_pop(L,1);
	 else {
	 	 printf(">> %s %i\n",getstr(source),source->tsv.len);
	 	 lua_pushlightuserdata(L,ls);
	 	 lua_pushcclosure(L,gettoken,1);
	 	 lua_pushlstring(L,getstr(source),source->tsv.len);
	 	 lua_call(L,2,0);
	 }
}

However, this means that once a filter function is installed, it expects
to be called only as a filter and *not* to set a filter for another
file.

The following version of the token filter file is completely off the top
of my head (and it's getting late anyway...), so it will certainly need
much more polishing. But it seems to work and it shows the basic idea:

local _get
function FILTER(get,source)
	if get==nil then -- we are being called as a filter
		local line,token,value=_get()
		print("FILTER",line,token,value)
		return line,token,value
	else
	  _get=get -- we are being called to set a new filter function
		print("FILTER",source)
  end
end

As I am relatively new to this token filter business, others (especially
lhf, I would think) will certainly find a much more elegant solution.

-- 
cheers  thomasl

web : http://thomaslauer.com/start