Re: Is it possible to yield/resume the main thread?

[Stefan <ste@...>]

Am 04.07.2020 um 18:21 schrieb Francisco Olarte:
>> So far my conclusions are:
>> * Never run the main thread with anything other than lua_call or
>>   lua_pcall
> 
> IMO this is one thing which you have got wrong. I like to consider it
> as if the main thread is running as soon as you do lua_open. It is
> just not running lua code. lua_*call just make it run it. If you view
> it like this it is less confusing. When I start the REPL an sitting at
> the prompt the main thread is running, and waiting in C code. When I
> execute io.stdin:read() it is running too, and waiting in C code
> invoked from lua code. But it is running.

Ah yes - now I see. luaB_costatus always gives "running" for the main
thread, "dead" for a thread with no bytecode (even a fresh one) and
"suspended" for one with a function loaded.

>
> AAMOF, it you look at coroutine.runnig sources in lcorolib.c you'll
> find "  if (L == co) lua_pushliteral(L, "running");".
> 
>> * Never lua_yield anything other than the currently running coroutine
> 
> I think lua_yield() is not going to be a problem if you pass the
> proper argument, the state pointer you normally pass around all your C
> code. A lua_state carries a coroutine context, but the main one
> carries some other things. The fact that lua_yieldk contains the line
> "      luaG_runerror(L, "attempt to yield from outside a coroutine");"
> makes me think you would just get an error if you do this and the
> state you passed happens to be the main thread.

Since the argument to a C function is the currently running thread,
this is the only valid way, I think (probably what you mean, too):

int c_func(lua_State* L)
or
int c_func_k(lua_State *L, int status, lua_KContext ctx)
{
    ...
    return lua_yield(L, ...);
    or
    return lua_yieldk(L, ...);
}

> 
> I suspect what is happening is calling lua_resume on the main thread,
> is incorrect but unchecked. This leads to corruption of things which
> then lets you call lua_yield avoiding the checks.
> 
> In fact I see in your example totally normal results in all cases
> except the lua_resume case, where I consider everything after running
> lua_resume is meaningless, as that is incorrect, as the main thread
> cannot be suspended ( every attempt to yield fails ) and you can only
> resume a suspended coroutine ( but lua_resume does not seem to check
> that ).

I've actually used the lua_resume(main_L) / lua_yield(L) quite a bit
and it generally works (without other coroutines). I never
suspected that the main thread cannot be used as a coroutine.
LUA_USE_APICHECK also doesn't detect this.

> 
> And I am not too sure of the utility of the parameter version of
> coroutine.isyieldable(). I mean, only a running coroutine can yield.
> You can test a suspended coroutine to see if it could yield as soon as
> it starts running, but it can put itself on an unyieldable state ( by
> entering a non-yieldable C function) as soon as you resume it, but I
> suppose for some cases using external communication mediums it can
> work ( maybe pushing a yielding callback or a non yielding one on the
> coroutines stack because you know it is going to call that, things
> like this ).
> 
>> I.e., the answer to "Is it possible to yield/resume the main thread?" is no.
> 
> Correct. I thought you were trying to asses the problems of calling
> lua_yield in a C callback being unsafe (crashing), which is not, from
> what I see it is as safe as calling coroutine.yield in lua as long as
> you do not mess with states, the worst that can happen is you get an
> error somewhere.
> 
> 
> Francisco Olarte.
> 

lua_isyieldable seems to be wrong:

lua_State* MAINL = luaL_newstate();
printf("luaL_newstate() = %p\n", (void*)MAINL);
printf("lua_isyieldable(MAINL) = %d\n", lua_isyieldable(MAINL));
printf("lua_status(MAINL) = %d\n", lua_status(MAINL));
printf("luaL_openlibs(MAINL)\n");
luaL_openlibs(MAINL);
printf("lua_isyieldable(MAINL) = %d\n", lua_isyieldable(MAINL));
printf("lua_status(MAINL) = %d\n", lua_status(MAINL));
lua_State* NTHRL = lua_newthread(MAINL);
printf("lua_newthread(MAINL) = %p\n", (void*)NTHRL);
printf("lua_isyieldable(NTHRL) = %d\n", lua_isyieldable(NTHRL));
printf("lua_status(NTHRL) = %d\n", lua_status(NTHRL));

This prints with Lua 5.4:
luaL_newstate() = 0x55d6635ff2a8
lua_isyieldable(MAINL) = 1
lua_status(MAINL) = 0
luaL_openlibs(MAINL)
lua_isyieldable(MAINL) = 1
lua_status(MAINL) = 0
lua_newthread(MAINL) = 0x55d663605d78
lua_isyieldable(NTHRL) = 1
lua_status(NTHRL) = 0

but with Lua 5.3:
luaL_newstate() = 0x5644debdf2a8
lua_isyieldable(MAINL) = 0
lua_status(MAINL) = 0
luaL_openlibs(MAINL)
lua_isyieldable(MAINL) = 0
lua_status(MAINL) = 0
lua_newthread(MAINL) = 0x5644debe6688
lua_isyieldable(NTHRL) = 0
lua_status(NTHRL) = 0

Maybe it is a side-effect of this change:

Lua 5.4.0  Copyright (C) 1994-2020 Lua.org, PUC-Rio
> coroutine.isyieldable(coroutine.create(function() end))
true

Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
> coroutine.isyieldable(coroutine.create(function() end))
false