Re: Patchless modification of Lua source code

[Sean Conner <sean@...>]

  Having consumed an inordiate amount of food earlier today [1] and
everything is fairly quiet and not much is going on, I thought I would
investigate this matter a bit further.  

  So without further ado ... 

It was thus said that the Great Viacheslav Usov once stated:
> On Mon, Nov 19, 2018 at 10:10 AM Dirk Laurie <dirk.laurie@gmail.com> wrote:
> 
> 
> > 1. All external names used by Lua are declared in header files.
> > 2. If such a name has been resolved by an object file or library
> > linked earlier, the linker will be satisfied and will not replace it.
> > 3. Any .o files needed by the linker will be built automatically from
> > .c files by the Makefile rules.
> >
> 
> The standard has no concept of "linked earlier" and consequently no primacy
> of "names linked earlier", and the entire program, including all the
> libraries, shall have a function with a given name defined at most once.
> Per clause 4/2, "If a ‘‘shall’’ or ‘‘shall not’’ requirement [...] is
> violated, the behavior is undefined".

  I'm not convinced.  I'm quoting the C99 standard.  First:

	5.1.1.1:

	1 ...  Previously translated translation units may be preserved
	  individually or in libraries....  Translation units may be
	  separately translated and then later linked to produce an
	  executable program.

  All this says is that compiled files (or "translation units" if you want)
can be saved as files, or stored in a "library" (which is not defined in the
standard, but is probably left undefined to give leeway to the
implementation) to avoid unnecessary compilation.

	6.2.2:

	2 In the set of translation units and libraries that constitutes an
	  entire program, each declaration of a particular identifier with
	  external linkage denotes the same object or function....

  So here we see that a program can consist of individual translation units
and "libraries".  This also says that each external declaration to a given
name will eventually resolve to the same object or function.  Now, onto the
point that appears to have the concept of "linked earlier":

	5.1.1.2:

	8 All external object and function references are resolved.  Library
	  components are linked to satisfy external references to functions
	  and objects not defined in the current translation.  All such
	  translator output is collected into a program image which contains
	  information needed for execution in its execution environment.

  I interpret this to mean, "if the given object files do not include an
object or function with external linkage, then said objects or functions can
be pulled from a "library".  So let's go over the critical step in the
procedure:

	make linux -e "LUA_O=lua.o myctype.o mylex.o"

  Going through the makefiles for Lua 5.3, I find that LUA_O is defined as:

	LUA_O=  lua.o
  
  Untangling the makefile (and I'll be following the Linux build since
that's what I'm using, but the others are similar) that when I do a

	% make linux

the build will make the targets defined in ALL_T, which is defined as LUA_A,
LUA_T and LUAC_T.  These are, respectively:

	LUA_A=  liblua.a
	LUA_T=  lua
	LUAC_T= luac

  These, in turn, depend upon 

	lua      : lua.o
	luac     : luac.o
	liblua.a : <object files except for the two listed above>

  This eventually will turn into the command line:

	gcc -std=gnu99 -o lua lua.o liblua.a -lm -Wl,-E -ldl -lreadline -lncurses

  On Unix, files with a '.a' extension are a "library".  The "-l" option is
just a shortcut to specifying other libraries one may link against, and
expanding everything out results in:

	gcc -std=gnu99 -o lua lua.o liblua.a /usr/lib/libm.so -Wl,-E /usr/lib/libdl.so /usr/lib/libreadline.so /usr/lib/libncurses.so

(also on Unix, files with a '.so' extension are also a "library" but the
difference between the two is not important for this disussion)

  Try to compile lua without specifying liblua.a and you get complaints
about missing external references (the first one is lua_sethook() by the
way).  Adding back liblua.a and everything compiles, since the missing
functions are found in a library per 5.1.1.2/8.

  Now, to the command in question:

	% make linux -e "LUA_O=  lua.o myctype.o mylex.o"

  This will be (effectively) expanded out to:

	gcc -std=gnu99 -o lua lua.o myctype.o mylex.o liblua.a /usr/lib/libm.so -Wl,-E /usr/lib/libdl.so /usr/lib/libreadline.so /usr/lib/libncurses.so

The file 'mylex.c' will presumedly provide definitions for the following
functions:

	luaX_init()
	luaX_setinput()
	luaX_newstring()
	luaX_next()
	luaX_lookahead()
	luaX_syntaxerror()
	luaX_token2str()

The three "translation units" provide a number of external functions---the
missing ones will therefore be pulled in from liblua.a, so in this case,
yes, there *IS* a form of the concept of "linked earlier".  The key sentence
from 5.1.1.2/8 is: 

	Library components are linked to satisfy external references to
	functions and objects not defined in the current translation.

  I would claim that lua.o, myctype.o and mylex.o constitude "the current
translation(s)" and that the functions in mylex.c will be used and the ones
that exist in the liblua.a "library" will not be, given the order in the key
sentance above.

> Note that what you do may fail in weird ways even with toolchains that do
> have the notion of "linked earlier" - precisely because something has been
> linked earlier. Your application may be using the "modified" functions,
> while Lua itself and the other libraries use the "original" functions. This
> would be a special case of generally undefined behaviour.
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Funny you mention that---I checked the C99 standard and found *nothing*
about this.  It's not unspecified, it's not undefined, it's not
implementation defined, its' not locale specific, *nothing*.  So yes, I
think this might be charitably noted as "generally undefined behavior" but
the C99 standard says nothing else about it.

  But given the wording of 5.1.1.2/8, I don't think your concern is
valid---translations units are searched prior to libraries for external
references so any reference to a function X found in a translation unit,
regardless of where it's used (in the current translation units or a
library), will be used.

  -spc (If at this point you don't think this interpretation holds, I'd like to
	hear the argument against it)

[1]	Okay, technically yesterday by the time this is going out, but I
	still consider it the same day.  Also, today is Thanksgiving Day in
	the United States of America, a holiday [2] typically celebrated by
	eating a large meal with friends and family and falling asleep
	shortly thereafter.

[2]	It's this weird quasi-religious/secular holiday were the purpose is
	to give thanks for what you have to whatever deity or non-deity of
	your (singular or plural) choice.