Re: Execution of bytecodes

[Sean Conner <sean@...>]

It was thus said that the Great Karel Tuma once stated:
> Excerpts from Sean Conner's message of 2015-10-03 01:02:26 +0200:
> >     const unsigned char c_blah[] = { ... };
> 
> Yes, that's how it's normally done. Alternate route is to write linker
> script and construct ELF .o object files directly - in rare cases when
> generating A LOT of data this way, and C compiler speed would be an issue.
>  
> > On modern Unix systems, const data like this is kept in memory marked as
> > read-only.  If there is memory pressure on the system as a whole and memory
> > needs to be "paged out", then such read-only sections can just be dumped and
> > not written to the paging system since if the pages are needed, they can be
> > paged back in directly from the executable on disk.
> 
> Yes, but again, premature optimization on modern systems.

  I *HATE* the phrase "premature optimization" as it's too dismissive and
smacks of "don't bother with *any* optimization because you're not clever
enough and you'll end up doing it wrong" (in my opinion).  The phrase
"premature optimization" comes from Knuth:

	Programmers waste enormous amounts of time thinking about, or
	worrying about, the speed of noncritical parts of their programs,
	and these attempts at efficiency actually have a strong negative
	impact when debugging and maintenance are considered. We should
	forget about small efficiencies, say about 97% of the time:
	premature optimization is the root of all evil. Yet we should not
	pass up our opportunities in that critical 3%.

  In *this* case, this "premature optimization" comes at no cost to
debugging *OR* maintenance of the program.  It's just *one* extra keyword
that says "this data should not change" [1] and the benefits, although
remote, are there.  The type of optimization Knuth is warning against is
stuff like: [2]

	Optimize some silly code I wrote to save a few memory bus cycles.

where the code in question is basically:

	+#define MASKED_FLAG_1	'1'
	+#define MASKED_FLAG_2	'2'

	-static const char maskflag1[] = "1";
	-static const char maskflag2[] = "2";

	-	const char *p_flag;
	+	int         flag;

	-	p_flag = maskflag1;
	+	flag = MASKED_FLAG_1;

	-	p_flag = maskflag2;
	+	flag = MASKED_FLAG_2;

		# about a dozen more variations on the above

	-	length=snprintf("%s",
	-		p_flag
	+	length=snprintf("%c",
	+		flag

  And that was it. [3]  *THAT'S* the type of optimization Knuth was warning
about.  Pointless changes that may or may not have an actual measureable
effect or that complicate the code needlessly.

  Sorry about that---hot button topic.

> >     1,045,919    uncompressed Lua bytecode
> >       937,183    compressed Lua bytecode
> >       921,587    compressed Lua source code
> > 
> 
> Don't do this. Use UPX with LZMA if the point is to reduce size of the whole
> result on disk (interpreter+uncompressed script).

  I never heard of this, and did some reading up on it.  In my case, it
won't work since we use a platform not supported by UPX. 

> Memory usage is essentialy
> the same here, even with very large scripts - the polluted heap from
> decompressor gets eaten by Lua heap during runtime anyway.

  It seems to pack the executable as whole, and not just portions of it. 
And while it can decompress in place, in general, it does extration to an
external file.  

  -spc 
 
[1]	Whether the compiler or system can enforce that is a different
	matter, but this at least lets the compiler check (to the extent
	that it can) that no code changes the data.

[2]	Actual check-in message.

[3]	I've come to the conclusion that the previous developer just assumed
	the code would be running a 1MHz 6502 with about 4K of RAM, never
	mind it being a 64-bit SPARC architecture with a dozen or more cores
	with gigabytes of RAM the way he kept on "optimizing" the code.