Re: helper threads layer

[Javier Guerra <javier@...>]

On Thursday 09 March 2006 1:16 pm, Vijay Aswadhati wrote:
> This may be the beginning of something new and exciting!!

i hope so! either because it's accepted, or because somebody does something 
much better after seeing this as a proof of concept.

> A few concrete examples would be of great help. I am particularly
> interested in how one would go about creating a singleton timer

this is precisely what i did as a testbed for the library.  here's the full C 
code:

#include <sys/time.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>

#include "lua.h"
#include "lauxlib.h"

#include "helper.h"

typedef struct timer_udata {
	struct timeval tv;
	int ret;
} timer_udata;

static int timer_prepare (lua_State *L, void **udata) {
	lua_Number t = luaL_checknumber (L, 1);
	timer_udata *td = (timer_udata *)malloc (sizeof (timer_udata));
	if (!td)
		luaL_error (L, "can't alloc udata");
	
	td->tv.tv_sec = (int) t;
	td->tv.tv_usec = (t - td->tv.tv_sec) * 1000000;
	
	*udata = td;
	
	return 0;
}

static int timer_work (void *udata) {
	timer_udata *td = (timer_udata *)udata;
	fd_set fd_a, fd_b, fd_c;
	FD_ZERO (&fd_a);
	FD_ZERO (&fd_b);
	FD_ZERO (&fd_c);
	
	printf ("before select\n");
	td->ret = select (0, &fd_a, &fd_b, &fd_c, &td->tv);
	printf ("after select\n");
	
	return 0;
}

static int timer_finish (lua_State *L, void *udata) {
	timer_udata *td = (timer_udata *)udata;
	
	if (td->ret < 0)
		luaL_error (L, strerror (td->ret));
	
	free (td);
	
	return 0;
}

static task_ops timer_ops = {
	timer_prepare,
	timer_work,
	timer_finish
};

int luaopen_timer (lua_State *L);
int luaopen_timer (lua_State *L) {

	helper_init ();
	add_helperfunc (L, &timer_ops);
	return 1;
}


as you can see, timer_prepare() gets the Lua parameters (a single number), 
allocates a private structure and fills it with a timeval (with sec and usec 
fields).  timer_work() runs in the background, so it can't touch the Lua 
space; it gets the timeval struct and does a select() to wait some time. 
timer_finish() verifies the return value of the select(), and frees the 
udata.  if there was any final result (like on a read funcion), it would be 
pushed in the Lua stack.

note that since it defines just a single function, it's returned in the stack 
by luaopen_timer(), instead of building a package.

this is how i tested it from Lua:

require "helper"
timer = require "timer"

q1=helper.newqueue()
q2=helper.newqueue()
print ("queues:", q1, q2)

th=helper.newthread (q1, q2)
print ("thread:", th)

tsk=timer(10)
print ("task:", tsk)

q1:addtask (tsk)
t2=q2:wait()
print ("t2:", t2)

helper.finish (t2)
print ("end")

----------------------------

th is the helper thread.  it uses q1 and q2 as input and output queues, 
respectively.  as soon as it's created, it tries to get a task from q1; but 
since it's empty, it's blocked.

the timer() function doesn't do the 'work' immediately, it just creates and 
returns a 'task' (a lightuserdata).  it also calls the timer_prepare() 
function.

when the task is added to q1, the thread gets it and executes the timer_work() 
function.  the Lua code continues to run in the 'main' thread.

the q2:wait() function blocks until there's a task in q2.  when timer_work() 
ends, the task is pushed to q2 and the Lua code unblocks.  t2 gets the task, 
it should be equal to tsk (that's why i used lightuserdata).  the 
helper.finish() call gets any result (none in this example)

> instance using this package and how the main dispatch loop would
> look like in the "C" or "C++" world.

my idea is that the main dispatch loop would be Lua code, not C.

i haven't written one yet, but this would be the general idea:

wrap all functions that return a task, like this:

local _read = read
function read (file)
    return helper.finish (yield (_read (file)))
end

and in the dispatcher:

while true do
    local tsk_done = out_q:wait()
    local co = coros [tsk_done]
    local tsk_blk = coroutine.resume (co, tsk_done)
    coros [tsk_blk] = co
    in_q:addtask (tsk_blk)
end


> On a frivolous note, I would probably name this package as
> 'concurrency' instead of 'helper'.

i'm notoriously bad at picking names for my code... any explanation why you 
think 'concurrency' is better than 'helper' ?

-- 
Javier

Attachment: pgpAmgeOddP_n.pgp
Description: PGP signature