lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


For such use, I see only a safe way: creating a pool of worker processes to handle concurrency, each process running its own Lua engine. The service the requests with a listener process that will schedule the requests to one of the available workers (which can be prestarted for just a few ones, and stopped when no lonner used for some time of each time they have serviced a maximum of N requests or a maximum time after completing one). If there are no idle worker available, spawn a new one (if you can with existing resources), or delay the request by putting it in a waiting queue, or abort the request from the queue if it has been in queue for too long.
You then need a scheduler. Such system exists as webserver extensions (e.g. FastCGI for Apache or IIS).

Given that processing each request will be isolated in their own process, it can be crashed in panic isolatedly without affecting other concurrent workers, and the scheduler will relaunch new workers on demand. The scheduler can detect such worker crash and return an 500 SERVER error to the client without ever revealing private data from other workers for concurrent requests.
This makes things simpler than managing "safe" multithreading between threads, where you don't know exactly what they would be doing internally (notably for workers running on a shared server, and instaciated by random users submitting their own Lua scripts).

If you want absolute security, each worker should be terminated after completing each request, and then restarted immediately to service other requests, unless all these requests are from the same identified user within the same security realm and limits set by your accounting administration (in which case the worker may be reused as long as the user needs it and stopped only after some time limit: this saves many unnecessary process recreations which could be very costly on a server to reinitialize a new Lua instance ready to service a single request, when your server is scaled with the intent to be usable to serve thousands requests or more each second from a measurable sert of users).


Le sam. 16 mai 2020 à 14:49, Massimo Sala <massimo.sala.71@gmail.com> a écrit :
Hi

I implemented a memory allocator.
To activate it, I call lua_newstate passing as arguments
- the pointer to the new allocator
- the pointer to a custom struct, of type tMemoryAccounting

The struct keeps informations necessary to our allocator algorithm.

Questions:
1) Concurrency

Our software can instantiate many instances via lua_newstate()
Each state shares the same allocator routine.
Each one has its own private struc.

Does Lua call the allocator with some sort of synchronization,
or should I protect the accesses to the struct (read and write) ?

Something like...
static void * _lua_alloc(void *ud, void *ptr, size_t osize, size_t nsize)
{
    tMemoryAccounting * pMemoryAccounting = (tMemoryAccounting *) ud;
    lua_lock(pMemoryAccounting->L);
    ...
    lua_unlock(pMemoryAccounting->L);
}

Should I add a  lua_State *L  to tMemoryAccounting ?
Am I missing a lua_get_current_state(void) ?


2) handling exceptions in the allocator

The new allocator in specific circumstances decides to deny its operations, allocations and free
(also free because it checks for possible corruptions of the memory pointers).

My requirement is: letting Lua fails the current API with the error LUA_ERRMEM.
For example: we can decide, at a certain moment, the allocator have to fail allocations above 1 Mb. For stress-testing purposes, or for other reasons.
If so, the Lua routines should return an error to our software, which ends gracefully.

If I understand correctly:
- the allocator cannot call luaD_throw, because it calls atpanic and aborts the program;
- cannot use setjmp: jumping out of the allocator, all the cleanup code of Lua current routine (destructors, stack, etc..) is skipped.

The problem can be split in two sub-problems:
- how can the allocator set the Lua error state?
- how can the allocator stop Lua's current routine processing and return to the caller?

For requests of new allocations / realloc, the allocator can return NULL and set, for example   L->status = LUA_ERRMEM;    ?
What about free ?

M

_______________________________________________
lua-l mailing list -- lua-l@lists.lua.org
To unsubscribe send an email to lua-l-leave@lists.lua.org
_______________________________________________
lua-l mailing list -- lua-l@lists.lua.org
To unsubscribe send an email to lua-l-leave@lists.lua.org