Re: Google Summer of Code 2013

[Paul K <paulclinger@...>]

Hi Steve,

> So candidates would be, creating bindings to existing libraries,
> taking an existing project as a base and building on it (e.g. using
> Paul K's excellent ZB studio and implementing a new debug back-end).

Thanks for bringing this up. Maybe not exactly what you had in mind,
but I've been wondering how much work would it be to provide more
complete debugging support in the Lua engine. I've seen clever
proposals from Dan Tull [1] and Thomas Jericke [2] (although I think
only Dan's patch was posted) that add a new opcode to support
breakpoints, but I would be looking for something a bit simpler and
possibly slightly more generic. The breakpoint support helps with
breakpoints (obviously), but requires additional work to support "step
over" and "step out" commands and doesn't help with breaking a running
application (although this can be emulated  with a count hook and
socket.select for TCP-based debuggers).

What if we propose something along the lines of having a simpler
mechanism that doesn't introduce new opcodes? It seems like most of
the time for debug hook processing is spent in luaD_callhook/luaD_hook
functions (not including the execution of the hook itself). For a
tight loop the difference can be 10 times (as measured on the
following code):

local l = 0
debug.sethook(function() l = l + 1 end, "l")
local s = os.clock()
local function DoSomething(x)
  x = x * 2
end
for outer=1,1000 do
  for inner=1,1000 do
    DoSomething(inner)
  end
end
print(os.clock() - s, l)

So, what if we add checks for breakpoints and stack depth that would
run before the actual hook processing (and are expected to be simple
and fast)? This would require sethook to be extended with "b"
(breakpoint) and "s" (stack) hooks and these would trigger a
corresponding "breakpoint" and "stack" types of callbacks. "stack"
would be a one-time callback and would take a relative parameter: 0
would mean staying at the same stack level ("step over"), and -1 would
mean going one level up ("step out"); other values would be possible
(semantics of going across resume and yield calls would need to be
discussed). "breakpoint" would take a line number and a filename
(probably in the same format as what's reported by getinfo). The new
logic could check for the line number match first (as this only
requires one lookup in the breakpoint table) and then do filename
processing if the line number matches. This is still slower than the
opcode processing, but I would expect this patch to be much simpler
and portable enough to be applied to Lua 5.1, Lua 5.2, as well as
LuaJIT.

One tricky part would be storing hook callbacks as the current
mechanism allows for only one hook. One simple possibility is to keep
using the current approach and to always store the last hook; This
means that if you do sethook(myhook, "l") and then sethook(myhook,
"b", 10, "file.lua"), then myhook will be called in both cases (which
is not very different from the current mechanism; sethook(nil, "b",
10, "file.lua") would remove the breakpoint hook and sethook(nil)
would remove all hooks as it currently does).

I'd be also interested in having a "timepoint" hook, which would be
called after a specified number of seconds (fractional), to implement
timeboxing and periodic "select" calls to suspend a running
application, but there may be challenges with cross-platform timers.

So, the project would be to "Develop an extension of existing debug
hook for low overhead breakpoints and step commands that can be used
with Lua 5.1, Lua 5.2 and LuaJIT." I think it can be immediately
useful, fully compatible with the existing code, sufficiently
interesting, and likely doable in the available timeframe. I'll be
happy to work on integrating it with Mobdebug to provide a front-end
for it.

[1] http://article.gmane.org/gmane.comp.lang.lua.general/69271
[2] http://lua.2524044.n2.nabble.com/Interests-in-a-breakpoint-hook-breakpoint-op-patch-td7484699.html

Paul.

On Tue, Feb 26, 2013 at 9:20 AM, steve donovan
<steve.j.donovan@gmail.com> wrote:
> On Tue, Feb 26, 2013 at 12:45 AM, Eric Wing <ewmailing@gmail.com> wrote:
>> So how about it?
>
> OK, now my understanding is that it should be a fairly well-defined
> project, enough to challenge a bright college student for a month or
> so.  So that excludes stuff which (as Sean rightly points out) are
> essentially social in nature, and ill-defined.
>
> So candidates would be, creating bindings to existing libraries,
> taking an existing project as a base and building on it (e.g. using
> Paul K's excellent ZB studio and implementing a new debug back-end).
>
> I remember that a sponsoring organization must exist and us cats got
> confused with that requirement.
>
> steve d.
>