Re: Execution of bytecodes

[Sean Conner <sean@...>]

It was thus said that the Great Marc Balmer once stated:
> 
> Now I wrote a script that will convert bytecode to a C character array and
> emit this with some glue code to a C compiler (clang in the case of the
> example).  This means that the bytecode sits there, directly in the
> binary, as a static character array that will never go away and which
> never is freed.  So in this case I am wasting memory, because I keep two
> copies of the bytecode in memory.  So with this usage pattern (and whether
> it is a sane one is a different question) it would be desireable to not
> copy everything into the interpreter, but just use it.

  I do this for some programs at work [1].  The character arrays are
declared as:

	const unsigned char c_blah[] = { ... };

On modern Unix systems, const data like this is kept in memory marked as
read-only.  If there is memory pressure on the system as a whole and memory
needs to be "paged out", then such read-only sections can just be dumped and
not written to the paging system since if the pages are needed, they can be
paged back in directly from the executable on disk.

  So in one sense, yes, it is taking memory.  But in another sense, once
it's no longer needed and the space does need to be reclaimed, it will be
reclaimed without wasting space in swap.

  Now, if you are still concerned abount memory usage, then you might want
to check out two messages I wrote last year, giving some figures:

	http://lua-users.org/lists/lua-l/2014-06/msg00433.html
	http://lua-users.org/lists/lua-l/2014-06/msg00434.html

  One other thing I do when I embed Lua modules written in Lua in an
executable is to compress the script prior to embedding [2].  I thought I
posted some other results, but I didn't so I'll do it here.  I did some
additional tests at the time, and I found that the non-compiled Lua source
compressed much better than the compiled Lua bytecode.  Given that the
"memory" will be wasted anyway, it might be better to compress the actual
source than to try using bytecode (compressed or not).  Some executable
sizes (all from a 32-bit Linux system; I don't have quick accesss to the
64-bit system right now):

	1,045,919	uncompressed Lua bytecode
	  937,183	compressed Lua bytecode
	  921,587	compressed Lua source code

  The differences will probably be bigger on the 64-bit system.  

  -spc

[1]	All modules used for our stuff built into a custom Lua interpreter.
	It makes installing much easier as it's only one file.

[2]	I wrote a custom loader that will uncompress the data as it's being
	loaded.  It was pretty straightforward using zlib.