Re: Trace compilation from embedded LuaJIT

["Robert G. Jakabosky" <bobby@...>]

On Wednesday 03, Douglas Creager wrote:
> Hi all,
> 
> I have a quick question about how much hot JIT action you'll get when
> embedding LuaJIT within another application.  My understanding is that
> LuaJIT's heuristics look for hot traces by identifying loops within the
> Lua code.  But what happens if the loop is in the host C application?  So
> for instance, I've got a data processing app where I'm allowing the user
> to do some custom processing by providing a Lua function.  That's going to
> be a hot function, since I'm processing many many records, so I'd like to
> have that function trace-compiled down into machine code.  [As an aside,
> this is a good use case for a possible lua_pushcdata function on the C
> side of things, to get a nice FFI cdata object into the Lua code from the
> host app.  But I digress.]
> 
> Are there any constructs that would let me accomplish this?  In stock Lua
> I'd just call the user's customization function using lua_pcall, but then
> there's no Lua loop to cause LuaJIT to start recording traces.
> 
> I've also tried using a coroutine, where I pass in each record using
> lua_resume:
> 
>   while true do
>     local more, record = coroutine.yield()
>     if not more then return end
>     process(record)
>   end
> 
> This gives me a loop, which will be executed once per record.  But
> unfortunately coroutine.yield is NYI, so I don't get a nice tasty compiled
> trace.
> 
> Any other things I can try?  Or is this too far outside of LuaJIT's
> intended use cases?

How about moving the data processing loop into Lua (for both the custom Lua 
callbacks and any other C callbacks).

while true do
  local record = get_next_record()
  if not record then return end
  -- call call all c/lua per-record callbacks
  for i=1,#callbacks do
    callbacks[i](record)
  end
  -- cleanup record
  cleanup_record(record)
end

If you are worried about the speed when not using LuaJIT, then have another 
version of that loop written in C.

-- 
Robert G. Jakabosky