Re: embedded lua: invocation overhead?

[David Olofson <david@...>]

On Friday 13 October 2006 19:16, Gabor Szokoli wrote:
[...]
> - What can we expect in such "impulse" performance?

I haven't benchmarked this, from studying the code, it seems like you 
might be better off "warping" the design, that is, staying in Lua 
most of the time, and calling C functions as needed.


> - What are the things I can trade off for startup speed? I'm sure I
> won't need a garbage collector for example, no sustained script
> execution.

Well, what I've seen of the Lua code (which is quite a bit, as I've 
studied it as part of my research for EEL), it doesn't exactly look 
like it's doing anything it doesn't need to do.

As to the GC, I think you *do* need it regardless. If you just disable 
it, most code that actually does anything will result in massive 
memory leaks, because there is nothing at all to reclaim memory. 
You'd have to replace the GC with some other form of memory 
management - and I don't think there is anything reasonably simple 
that has a chance of being faster than the current GC.


> - Would startup time benefit from using LuaJIT Ahead of Time
> compilation?

It certainly speeds up execution, but I don't know how it affects 
entering and leaving Lua.


> - Can I make sure no script can hurt the host process?

Actually, you'd probably have to do some stupid things in your 
application to enable scripts to hurt the host at all. Keep in mind 
that Lua is not some fancy portable assembly language, like C or 
C++. ;-)

Of course, there is always the good old "loop forever" and "consume 
infinite amounts of memory" holes, but I would think you can deal 
with those without much trouble, if need be.


> - Can I destructively interrupt the "parse" (or whatever the
> equivalent of dofile is for an in-memory bytecode-compiled script
> which I am sure I can cache and reuse) function when a safety timer
> expires?

Something built around lua_sethook() should do the trick, I think...


> Our current contenders for the role in order of preference:
> Lua, "roll our own" (possibly with XIDEK), and Squirrel.

Rolling your own can be quite a big job, depending on what you need... 
Getting things fast tends to complicate matters a great deal, and of 
course, complexity explodes as you throw a substantial feature set 
into the mix.

Don't know about XIDEK, but it looks like it might save you a great 
deal of work. Apparently, it comes with a "simple" bytecode VM. Not 
sure what that means exactly, but generating efficient bytecode is 
not trivial, and nor is executing bytecode quickly. Then again, 
certain types of domain specific languages pretty much inherently 
generate simple and fast VM code, simply because there aren't all 
that many ways to get the job done.

Of course, for my pet project EEL, I just wrote it all from scratch in 
plain C - but that's just because my mind is twisted enough that I 
like toying with this kind of stuff. Oh, and I needed something lean 
and mean for scripting hard real time applications. :-)


> Let me know if you know of any other likely competitors :-)

If you're a bit adventurous, you could have a look at this EEL thing 
I'm talking about every now and then:
	http://eel.olofson.net/

It's used in embedded production systems, but it's nowhere near as 
mature as Lua. I've singlehandedly written most of it over a period 
of about two years while working on various other stuff, so what can 
you expect...!? ;-)

Why would EEL be of interest to you, then?

Well, it just happens that entering and leaving the VM quickly has 
been a priority, simply because EEL is inteded for "high cycle rate" 
real time applications. I didn't want to dictate a single "correct" 
way of doing things, so I aimed at making C calling EEL as fast as 
the other way around, if possible. (I think it's pretty close...)

Also, EEL is using reference counting for memory management, which 
means memory management is tighter and more predictable than with GC, 
if perhaps not as efficient as GC when running big batches of code. 
(*) Maybe this happens to be a more appropriate model for your 
application?

(*) There are different opinions on that, actually. Some claim
    that GC is suboptimal on modern CPUs because the larger
    memory footprint result in more CPU cache misses. Refcounting
    reclaims memory sooner (or rather instantly, in the basic
    text book version), which decreases the risk of hitting "cold"
    memory after allocations.


//David Olofson - Programmer, Composer, Open Source Advocate

.-------  http://olofson.net - Games, SDL examples  -------.
|        http://zeespace.net - 2.5D rendering engine       |
|       http://audiality.org - Music/audio engine          |
|     http://eel.olofson.net - Real time scripting         |
'--  http://www.reologica.se - Rheology instrumentation  --'