Re: Pre-emptive round robin scheduler around Lua VM

On Tue, Apr 1, 2014 at 4:30 AM, Milind Gupta <milind.gupta@gmail.com> wrote:

Hi,
Please can you share the example that would not work. The above example as you mentioned seems to work fine.

Milind

On Tue, Apr 1, 2014 at 4:13 AM, l m <ml@gizal.org> wrote:
hi,

it would not work for coroutines

if there is more than one level of coroutine, i don't see a simple solution

with only one this seems to work ( with the issue that hook count is
not global ) :

#include <lua.h>
#include <lauxlib.h>
#include "src/lstate.h"

lua_State* resume_thread;
int resume_main;

void hookFunc(lua_State *L, lua_Debug *ar) {
lua_State* M = L->l_G->mainthread;
if(L != M) {
if(resume_thread == M)
resume_thread = L;
M->hookcount = 1;
}
resume_main = 0;
lua_yield(L, 0);
}

int main() {
lua_State *L1, *L2, *N1, *N2;

L1 = luaL_newstate();
L2 = luaL_newstate();

luaL_openlibs(L1);
luaL_openlibs(L2);

lua_sethook(L1, hookFunc, LUA_MASKCOUNT, 500);
lua_sethook(L2, hookFunc, LUA_MASKCOUNT, 500);

luaL_loadstring(L1, "function co() while true do for i=1,100 do
print('B') end coroutine.yield() end end"
" c = coroutine.create(co) coroutine.resume(c)"
" while true do for i=1,70 do print('A') end
coroutine.resume(c) end");
luaL_loadstring(L2, "while true do print('C') end");

N1 = L1;
N2 = L2;

for(;;)
{
resume_main = 1;
resume_thread = L1;
lua_resume(N1, NULL, 0);
if(N1 != L1 && resume_main) lua_resume(L1, NULL, 0);
N1 = resume_thread;

resume_main = 1;
resume_thread = L2;
lua_resume(N2, NULL, 0);
if(N2 != L2 && resume_main) lua_resume(L2, NULL, 0);
N2 = resume_thread;

}
}

/*

% ./a.out | uniq -c | head
100 B
70 A
21 B
125 C
79 B
70 A
44 B
125 C
56 B
70 A

*/

nicolas.

On 3/31/14, Milind Gupta <milind.gupta@gmail.com> wrote:
> Here is a crude round robin schedules I had tried but had not tried
> coroutines in the lua code. This scheme is working good for me:
>
> #include <stdio.h>
> #include <string.h>
> #include "lua.h"
> #include "lualib.h"
> #include "lauxlib.h"
>
>
> int flag = 0;
> lua_State *luaVM1, *luaVM2, *thread1, *thread2;
> int hookFunc(lua_State *L, lua_Debug *dbg);
>
> int main(int argc, char* argv[ ])
> {
> luaVM1 = luaL_newstate();
> luaVM2 = luaL_newstate();
>
> luaL_openlibs(luaVM1);
> luaL_openlibs(luaVM2);
>
> thread1 = lua_newthread(luaVM1);
> thread2 = lua_newthread(luaVM2);
>
> if (NULL == luaVM1 || NULL == luaVM2)
> {
> printf("Error Initializing lua\n");
> return -1;
> }
> char* prog1 = "for i = 1,5 do print('x'..tostring(i)) a=io.read()
> print(a) end";
> char* prog2 = "for i = 1,10 do print('y'..tostring(i)) end";
> // Create threads in each lua state
> // Now push the function into the stack of each thread
> luaL_loadstring(thread1,prog1);
> luaL_loadstring(thread2,prog2);
> lua_sethook(thread1,(lua_Hook) hookFunc,LUA_MASKCOUNT,4);
> lua_sethook(thread2,(lua_Hook) hookFunc,LUA_MASKCOUNT,4);
>
> lua_resume(thread1,0);
> flag=1;
> lua_resume(thread2,0);
> while(lua_status(thread1) || lua_status(thread2))
> {
> if (flag==0)
> {
> flag = 1;
> if (lua_status(thread2))
> lua_resume(thread2,0);
> }
> else
> {
> flag = 0;
> if (lua_status(thread1))
> lua_resume(thread1,0);
> }
> }
>
>
> lua_close( luaVM1 );
> lua_close( luaVM2 );
>
> return 0;
> }
>
> int hookFunc(lua_State *L, lua_Debug *dbg)
> {
> printf("Hook Called\n");
> if(flag==0)
> {
> printf("Flag0\n");
> return lua_yield(thread1,0);
> }
> else
> {
> printf("flag1\n");
> return lua_yield(thread2,0);
> }
> return 0;
> }
>
>
>
> On Mon, Mar 31, 2014 at 7:53 AM, Milind Gupta
> <milind.gupta@gmail.com>wrote:
>
>> I have my debug hook function as int and don't have issues although I
>> don't use coroutines. Here is an excerpt from Programming in Lua 2nd
>> Edition in the Threads chapter:
>>
>> "
>> Standard Lua cannot yield across C function calls. This restriction
>> implies
>> that a C function cannot suspend itself. The only way for a C function to
>> yield is
>> when returning, so that it actually does not suspend itself, but its

>> caller--which

>> should be a Lua function. To suspend its caller, a C function must call
>> lua_yield
>> in the following way:
>> return lua_yield(L, nres);
>> "
>>
>>
>> On Mon, Mar 31, 2014 at 7:46 AM, Dominique Torette <
>> Dominique.Torette@spacebel.be> wrote:
>>
>>> From 'lua.h', debug hooks are void functions:
>>>
>>>
>>>
>>> typedef void (*lua_Hook) (lua_State *L, lua_Debug *ar);
>>>
>>>
>>>
>>> Regards, Dominique.
>>>
>>>
>>>

>>> *From:* lua-l-bounces@lists.lua.org [mailto:lua-l-bounces@lists.lua.org]
>>> *On
>>> Behalf Of *Milind Gupta
>>> *Sent:* lundi 31 mars 2014 16:34
>>> *To:* Lua mailing list
>>> *Subject:* Re: Pre-emptive round robin scheduler around Lua VM

>>>
>>>
>>>
>>> Hi,
>>>
>>> One thing I have seen in the Programming in Lua book is that
>>> the Hook function in C should always have a yield in the return
>>> statement
>>> so the debug_hook function should not be void and the last statement
>>> should
>>> be return lua_yield. Does that get rid of the interactions?
>>>
>>> Regards,
>>> Milind
>>>
>>>
>>>
>>> On Mon, Mar 31, 2014 at 7:03 AM, Dominique Torette <
>>> Dominique.Torette@spacebel.be> wrote:
>>>
>>> Hi,
>>>
>>>
>>>
>>> I'm trying to build a kind of pre-emptive scheduler around Lua VM 5.2.
>>>
>>> The idea is to execute different lua_State in "parallel'.
>>>
>>> To give the illusion of parallelism, the idea is to run a thread for a
>>> given number of Lua VM instructions and then run the next thread in a
>>> list

>>> for this number of instructions and so on...

>>>
>>> From the LUA regular API, when starting the execution of a thread, it is
>>> not possible to control the number of Lua VM instructions to execute.
>>>
>>> But from the LUA debug API, it is possible to define a 'hook' to execute
>>> every COUNT of instructions.
>>>
>>> The first idea was to install such hook and perform the round-robin
>>> scheduling from the hook. The lua_State of active thread seems to be
>>> saved
>>> before running such hook.
>>>
>>> But I didn't found any means to overwrite the thread to be resumed from
>>> the hook.
>>>
>>> Then the idea was to 'yield' from the hook, back to the round robin
>>> scheduler.
>>>
>>> This seems to work in the simplest cases. But I've noticed some strange
>>> "intereferences" when thread use coroutines on its own.
>>>
>>>
>>>
>>> The Lua resume API has a 'from' parameter: int lua_resume (lua_State *L,
>>> lua_State *from, int nargs);
>>>
>>> But from the description, I didn't catch the how this param has to be
>>> used.
>>>
>>>
>>>
>>> I've written a small "C" main program to illustrate the concept and the

>>> "interferences"...

>>>
>>> In the program here below, the first iteration of the loop works as
>>> expected.
>>>
>>> Then the L[0] and its coroutine are resumed, but the hook is reached
>>> faster than for the first iteration and the L[1] is rescheduled.
>>>
>>> After L[0] will be never rescheduled, only L[1].
>>>
>>>
>>>
>>> Thanks in advance, Dominique Torette.
>>>
>>>
>>>
>>> #include "lua.h"
>>>
>>> #include "lauxlib.h"
>>>
>>> #include "lualib.h"
>>>
>>> #include "lstate.h"
>>>
>>> #include <stdio.h>
>>>
>>> #include <setjmp.h>
>>>
>>>
>>>
>>>
>>>
>>> lua_State* L[2];
>>>
>>> lua_State* S[2];
>>>
>>> int current;
>>>
>>> int previous;
>>>
>>>
>>>
>>> void debug_hook (lua_State *X, lua_Debug *ar)
>>>
>>> {
>>>
>>>
>>>
>>> printf("In debug_hook\n");
>>>
>>> if((X->l_G)->mainthread != X)
>>>
>>> {
>>>
>>> printf("from coroutine \n");
>>>
>>> }
>>>
>>> S[current] = X;
>>>
>>> current = (current+1)%2;
>>>
>>> lua_yield ((X->l_G)->mainthread,0);
>>>
>>> }
>>>
>>>
>>>
>>> int main(int argc, char * argv)
>>>
>>> {
>>>
>>> int status;
>>>
>>>
>>>
>>> L[0] = luaL_newstate();
>>>
>>> luaL_openlibs(L[0]);
>>>
>>>
>>>
>>> luaL_loadstring(L[0], "co = coroutine.create(function () while
>>> true do print('hi') coroutine.yield() end end) while true do
>>> print('hello')
>>> coroutine.resume(co) end");
>>>
>>> lua_sethook (L[0],debug_hook,LUA_MASKCOUNT,500);
>>>
>>>
>>>
>>> L[1] = luaL_newstate();
>>>
>>> luaL_openlibs(L[1]);
>>>
>>> luaL_loadstring(L[1], "while true do print('world') end");
>>>
>>> lua_sethook (L[1],debug_hook,LUA_MASKCOUNT,500);
>>>
>>>
>>>
>>> current = 0;
>>>
>>> S[0] = L[0];
>>>
>>> S[1] = L[1];
>>>
>>> for(;;)
>>>
>>> {
>>>
>>> printf("Resume 1\n");
>>>
>>> status = lua_resume (S[0],L[0],0);
>>>
>>> printf("status 1 %d\n",status);
>>>
>>> printf("Resume 2\n");
>>>
>>> status = lua_resume (S[1],L[1],0);
>>>
>>> printf("status 2 %d\n",status);
>>>
>>> printf("Reloop\n");
>>>
>>> }
>>>
>>> }
>>>
>>>
>>>
>>>
>>>
>>>
>>> [image:
>>> http://www.spacebel.be/wp-content/uploads/2011/06/image-sign-sbp.jpg]
>>>

>>> *Dominique Torette*

>>> System Architect
>>> Rue des Chasseurs Ardennais - Liège Science Park - B-4031 Angleur
>>> Tel: +32 (0) 4 361 81 11 - Fax: +32 (0) 4 361 81 20
>>> www.spacebel.be
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> E-MAIL DISCLAIMER
>>>
>>> The present message may contain confidential and/or legally privileged
>>> information. If you are not the intended addressee and in case of a
>>> transmission error, please notify the sender immediately and destroy
>>> this
>>> E-mail. Disclosure, reproduction or distribution of this document and
>>> its
>>> possible attachments is strictly forbidden.
>>>
>>> SPACEBEL denies all liability for incomplete, improper, inaccurate,
>>> intercepted, (partly) destroyed, lost and/or belated transmission of the
>>> current information given that unencrypted electronic transmission
>>> cannot
>>> currently be guaranteed to be secure or error free.
>>> Upon request or in conformity with formal, contractual agreements, an
>>> originally signed hard copy will be sent to you to confirm the
>>> information
>>> contained in this E-mail.
>>>
>>> SPACEBEL denies all liability where E-mail is used for private use.
>>>
>>> SPACEBEL cannot be held responsible for possible viruses that might
>>> corrupt this message and/or your computer system.
>>>
>>> -------------------------------------------------------------------------------
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> E-MAIL DISCLAIMER
>>>
>>> The present message may contain confidential and/or legally privileged
>>> information. If you are not the intended addressee and in case of a
>>> transmission error, please notify the sender immediately and destroy
>>> this
>>> E-mail. Disclosure, reproduction or distribution of this document and
>>> its
>>> possible attachments is strictly forbidden.
>>>
>>> SPACEBEL denies all liability for incomplete, improper, inaccurate,
>>> intercepted, (partly) destroyed, lost and/or belated transmission of the
>>> current information given that unencrypted electronic transmission
>>> cannot
>>> currently be guaranteed to be secure or error free.
>>> Upon request or in conformity with formal, contractual agreements, an
>>> originally signed hard copy will be sent to you to confirm the
>>> information
>>> contained in this E-mail.
>>>
>>> SPACEBEL denies all liability where E-mail is used for private use.
>>>
>>> SPACEBEL cannot be held responsible for possible viruses that might
>>> corrupt this message and/or your computer system.
>>>
>>> -------------------------------------------------------------------------------
>>>
>>
>>
>