luapi An api extension for Lua 4.0 (C) Marcelo Nicolet 2002 mnicolet@satlink.com MANIFEST doc/luapi.txt : this text include/luaex.h : declaration of some 'primary' Lua's api extensions include/luapi.h : declaration of types, macros and functions src/lapiex.c : extension to the 'primary' Lua's api src/luapi.c : luapi module hosttest/host.c : host C test program. hosttest/makefile hosttest/autor.3 : example command file for host hosttest/t.lua : lua module loaded by autor.3 LICENSE The license terms for luapi are exactly the same as for Lua 4.0 INSTALLATION This installation method is the one I used, and have the merit of no large scale changes to the original Lua distribution. 1 - Unpack this tarball in the same directory where your Lua 4.0 sources reside, so the .h, and .c main files go to include and src respectively, and hosttest will be made as a sibling of the previous. 2 - Get lapiex.c compiled and included in the liblua.a lib. The best way to get this accomplished is to modify lapi.c so it includes lapiex.c; also, I think it's good to place the #include directive at the very end of lapi.c If you don't like #includes of code files ( some people do ), you must a) modify lapiex.c to #include all the header lapi.c does, and modify the list of the objects in the Makefile so it is aware of the new object. b) paste the lapiex.c code directly into lapi.c For the 'include' method, modify the Makefile also, inserting an explicit target for lapi.o as follows lapi.o : lapi.c lapiex.c so if you make changes to lapiex.c, the whole thing will be made again. 3 - Get luapi.c compiled and included into its own library ... Modify the Makefile to - 'declare' a new library A= $(LIB)/luapi.a - add it to the all target all: $T $A - explicit luapi.a target dependence and command $A: luapi.o $(AR) $@ luapi.o If you are using gmake, don't forget the tab before $(AR) $@ luapi.o and the blank line after. 4 - To build the host test program I use Watcom 10.6 and an old version of gmake, in a QNX 4.25 env. The makefile provided copies some ../src modules to this directory to allow compilation for symbolic debugging ( some debuggers assumes that the source files referenced in the objects are local to the current directory .. ) To avoid large recompilations, the 'cp' command must be used in 'smart mode' ( copying only newer files ); modify the CPCMD variable to suit your cp options. Modify the Dbg variable to suit your compiler/linker/taste options for debugging, also emptying it. Modify CFLAGS and LFLAGS to suit your compiler/taste regarding warnings, output file name, include and library directories, etc When you run make, you will get two binaries host - linked with the libraries hostdbg - linked with the objects locally compiled, and using the libraries only for not resolved symbols Of course, you can add or delete 'local objects' as you need, simply adding or deleting them from the lists, and writing or deleting the corresponding targets. I will not document host/hostdbg usage in detail ( please see the source for that purpose - it's very simple ), but it allows lots of interactive Lua manipulation such as - loading of lua modules - get, set and deletion of globals on global deletions, it also recognizes global table deletions, and queries the user regarding which mechanism to use. This is great to test luapi_deltable and its options. - Explicit table creation Being the most convoluted topic, this is a separate option, and allows ( see the code to learn exactly how ) to recursively set a table member to a subsidiary one, or a pre-existent global, etc - Table operations Every time a table is 'in focus', it allows to - get a 'row' by key - add a 'row' ( index, value ) the 'row value' can be itself a table created 'on the fly' - delete a 'row' by key if the 'row value' is itself a table, all the options regarding table deletions are allowed - browsing it you can 'filter' the type of the elements to be showed during the browse - Function execution Every time a function is 'in focus', it allows you to execute it, passing as many parameters as you like, and showing the results. The execution itself can be made using the name of the function ( if it's global ) or the reference to allow test both methods. Tables and functions are 'in focus' as a result of every operation that retrieves a Lua object, i.e, getting a global, browsing or examining a table, and as a function result. The only case which is not considered 'active' is the C function parameters show, simply to not mess the display. Last but not least, one special argument sintax allows you to execute a set of commands stored in a file, repeating them n times. This is @n,commandfile, where the '@' signals the special option, n is the number of iterations you want, and commandfile is ... that. To create a command file, the best way is to do the things you would like interactively, recording your keystrokes as lines of an archive. This is specially true with regard to the newlines required to 'break' some options ... May be some day I will write a 'recording' option This feature is useful to two purposes: a) to check memory leaks. Initially, I wonder regarding the exact mechanism required to delete a table which allows the gc to collect those deleted objects ... In fact, I am not sure if the laborious mechanism used in deltable ( restarting the browsing until no more elements are left ) is really needed ... but it works You can create ( 'manually' or loading a module which does ) and delete a table repeteadly but the data segment of the program stabilizes after a few iterations. b) to 'time' some path of execution. By the way, any other argument to the executable is interpreted as a lua module to load. The whole thing is, in this respect, a nice demonstration of the reflectivity of Lua, and how luapi follows that paradigm. You can see how there are a short handful of functions which serve all purposes. That is not the result of careful design, rather the program was grown as new luapi functions and features went functional, and then, some code chunks reorganized because common functionality, put into focus by the api new capabilities. Enjoy ! MOTIVATION AND PURPOSE I was using Lua 3.2 as the configuring language for a very ambitious general accounting system. That was a success, even if the lua's usage is 'in extension' and not 'in deep'; i.e I was a Lua 'newbie' and limited its uses to the most simplistic ones. Also, there was the user question: you cannot face it with sophisticated and carefully sintax without risk of a module being rejected over and over. But this is another, generic question. Because I am focusing on Lua for some even more ambitious projects, it seems a Good Thing to migrate to Lua 4.0, at least because of separate 'interpreters' ( lua_States ). And there is a lot more ... When I saw the api differences among versions, and how I interspersed Lua api functions into my production code, and the code changes involved, I decided it was time to 'officialize' two main things - the first one, to generalize and standardize some ideas I used in the accounting system - the second, to make some draft ideas present in that system a 'second stage' generic Lua's api. Both aims are intimately related, and follows a single direction, mainly - to simplify the Lua interface from C. This has mainly to do with shortened lua stuff code ( which sometimes are a lot of lines without clear relation with the code purpose ) - to hide future changes in the api. This may be broken by future big changes not only in the api, but in the concepts underlying, but Waldemar, Roberto, and Luiz surprise us with very cool changes, so .. it's a matter of bet. - to retain ( surprisingly for me, also ) some 3.2 functionality The luapi code is surprisingly simple. This is not a merit of mine; the main credit must be for Lua authors. I simply put it to work, after a week of carefully study of some Lua internals, coding and testing. Documentation What it provides a) a comfortable mechanism to pass values from/to lua interpreter. This is mainly accomplished by the luav_ functions, and luapi_push/luapi_spval/luapi_pop/luapi_stack. The luav_ family relies on a circular buffer to obtain ( if needed ) memory for transparently pass values from/to lua, allowing you to pass NULL pointers from your C code, if you like. The main interest of this mechanism is that it is flexible: for some of the 'objects' involved, you can elect among allocate in the stack of your C function, or allows luapi to obtain the required memory. In this last case, the very important thing is that you are not concerned with heap management. Of course, care must be taken: this memory is as 'volatile' as a stack variable. Note that 'string values' are always 'copied' to allocated memory ( this simplifies the bussiness ), and that, when the 'value' comes from lua ( luapi_spval( ) ), lua_strlen( ) is used to get the length ( honoring the 4.0 ability to store arbitrary byte sequences in a string ); so if you are using lots of big strings, it may be wise to set CBUF_SIZE to some bigger than the current value, which is calculated to store 3000 luaval_t no matter how much padding your compiler puts between structure members. At a second level, the transparent value passing ( luapi_push() and luapi_spval() ) does two important things a.1) conversion 'type' from luav types to Lua types and viceversa a.2) access a special api extension function ( lapiex.c and luaex.h ) to push function and table 'objects' to the Lua stack. This allows you to reference 'anonymous' functions and tables from C code without worry. a.3) complete and simple table manipulation from C code. b) some important functions some of these are important only because they do stack management, and others because they provide 'high level' functionality In all what follows, usually the first parameter is a lua_State pointer, so it is not explicitly mentioned. b.1) luapi_getglob given a name, and maybe a luaval_t pointer, returns the result of lua_getglobal, in the luaval_t pointer passed, or in a new allocated one. Arguments: char * name, luaval_t * Returns: pointer to luaval_t, the one passed or an allocated one if NULL argument. NULL only if name == NULL; if the global requested does not exists, val->type would be LAPI_NIL b.2) luapi_setglob given a name, and a luaval_t pointer, it sets the 'name' to the 'value' pointed by luaval_t Arguments: char * name, luaval_t * val No check is made for 'nullity' of either argument Returns: void b.3) luapi_delglob given a 'lua object' ( luaval_t ) and/or a name, it searches the global names table and deletes the entry from it. If both name and obj are NULL, the function merely returns. If only an object is given, the function tries to guess the name. This applies only to tables and functions, because the 'identity' mechanism used to guess an object name is ambiguous for anything else ( see objname and luav_eq ). See also note regarding userdata. As a result, if only an object reference is given and it's not a table or function, delglob returns doing nothing. If the name or 'luaval_t' happens to reference a table, all the table elements are deleted ( see deltable doc ahead ) passing deltable a harmless flag. Arguments: char * name, luaval_t * obj Returns: void b.4) luapi_newtable creates a new table object, and optionally, populates it. the second argument ( name ), if not null, makes the table 'global', registering the name in 'gt' the function returns a reference ( luaval_t ) to the newly created table wich you can use to assign 'the table' as an anonymous one to anything you like. Population is done taking an array of 'luapairs_t' and calling addrow ( see below ) Arguments: char * name, luapair_t * p, int n name and p are optionals, if p is given, n must reflect the actual size of p. Returns: luaval_t * ref, reference to the new table this may be NULL in case lua_newtable does not push a table onto the stack ( ? ) b.5) luapi_deltable deletes a table referenced by a luaval_t pointer. this is done by 'browsing' every row of the table, and deleting it. The 'browsing' is restarted every time, to not mess the hashing functions. If the table being deleted is in the global namespace, the behaviour is determined by the gind argument: if it has the LAPI_GNAMDel bit set, the name is also deleted. If a 'row' to be deleted references a table ... the function checks if the table is not a global one, or else the gind argument has the LAPI_GMEMDel bit set; if some of these conditions are met, the member table is recursively deleted passing the same gind received. Note that delglob, if asked to delete a table, calls deltable with gind = 0, so b.1) no global tables referenced are deleted b.2) only the named table is deleted from the global table ... which is a delglob task ... Arguments: luaval_t * ref, int gind Returns: void b.6) luapi_addrow given a luaval_t pointer that references a table ( global or anon ), a luapair_t pointer and an integer that states the luapair_t size, it adds the rows described by the luapair_t array to the table ... Checks are made on the reference being NULL or having a type != LAPI_TAB, and luapair_t being NULL, returning zero. The n argument states a maximum ( to avoid segment violations ), the luapair_t array walk may be interrupted early if the 'key' member has type == LAPI_NONE. Arguments: luaval_t * ref, luapair_t * p, int n Returns: the number of rows added b.7) luapi_delrow same as addrow, but the keys referenced by each luapair_t are deleted from the table. Arguments: luaval_t * ref, luapair_t * p, int n Returns: the number of rows deleted b.8) luapi_getrow given a table and one luapair_t pointer, this function attempts to get the key member from the luapair_t, setting the val member of luapair_t to the result. Checks are made on the reference being NULL or having a type != LAPI_TAB, and luapair_t being NULL, returning NULL. Arguments: luaval_t * ref, luapair_t * p Returns: same pointer to p or NULL if some of the checks fails. b.9) luapi_tbrow given a table reference ( luaval_t ) and a luapair_t, it gives the next pair key/value from table, putting it in the luapair_t pointer. Checks that luaval_t is not NULL and has the type LAPI_TAB, else returns NULL. If luapair_t is NULL, a new one is allocated, and the key member set to LUA_TNIL, in fact starting a browsing. Arguments: luaval_t *, luapair_t * Returns: luapair_t * ( passed or allocated ) on success NULL in the case of check failure or exhausted browsing b.10) luapi_Gt returns a 'pointer' ( luaval_t ) to the current globals table Arguments: luaval_t * if this argument is NULL, one will be allocated Returns: the same luaval_t passed or the allocated one b.11) luapi_callobj calls the referenced object ( which must be of type LAPI_FUN ) with the arguments provided by the luaval_t arg array, optionally 'counted' by the n argument. See below the luapi_call function doc for details regarding arg and n. returns an array of luaval_t objects which are the function results, 'delimited' by a LAPI_NONE 'object' Arguments: luaval_t * funobj, luaval_t * args, int n Returns: on sucess, a luaval_t array, whose size is the number of call results plus one ( so even functions returning no results returns at least one element ) on failure ( local checks, lua errors ) returns NULL b.12) luapi_callfun same as callobj, but receives the function name rather than the reference ... b.13) luapi_call this is the function that both callobj and callfun invokes for argument 'parsing' and pushing. The important thing about it is the n argument interpretation if n is < 0, it assumes that the end of the arguments is signaled by an arg element having type LAPI_NONE, else n is the exact count, and any LAPI_NONE argument will be pushed as a lua nil. Arg may be NULL, in fact calling the function with zero args. Stackpos argument is passed directly to the stack function, see its doc below for a discussion. Arguments: luaval_t * args, int n, int stackpos Returns: luaval_t * array ( provided by stack( ) ) or NULL in case of error b.14) luapi_stack returns the stack content as a luaval_t array, terminated by a LAPI_NONE type object. the size of the array is determined by lua_gettop( L ) incremented by one ( so making place for the LAPI_NONE 'delimiter' ) The array is allocated from the circular buffer ... This function is used by callfun and callobj to get the function call results, and is very useful to get the arguments to a C function. Arguments: int stackpos Returns: luaval_t * stackpos pretends to be the lower stack pos where results or arguments start. Discussion: At initial develop stages, this argument was the 'lua function stack place' to help the function evaluate the number of elements it must pick from the stack. ( see the commented code ). But at a later stage, that usage was hidding results in some cases. At this moment in luapi usage ( see host.c ) there is no need to do strange arithmetic ... but some future use and/or advice may change things. So, even if not used, the argument stays there. b.15) luapi_objname given an object reference ( luaval_t ) searches the current globals table comparing the reference with every of the values; when and if an exact match is found, the name ( the key ) is returned. It's useful ( or exact ) only with tables and functions; used with strings or numbers, it may give spurious results ... Arguments: luaval_t * Results: the name ( a char * ) of the first object that exactly 'equals' the one passed, or NULL if not found. b.16) luav_tsize returns the number of values in a luaval_t array until a LAPI_NONE is found. Specially useful for arrays returned by the stack function. Arguments: luaval_t Returns: int, the number of elements in the array until the first LAPI_NONE is found, not counting it. A word regarding LAPI_ types: LAPI_NONE is defined as zero, so it's the 'default' type when a luaval_t object is allocated from the circular buffer, or one does a 'memset( lv, 0, sizeof lv )' on a C allocated array. All remaining 'types' are a 'function' of LUA 'types' to avoid clashing with LAPI_NONE, and are != 0. When pushing a LAPI_NONE luaval to the stack, it's interpreted as LUA_TNIL ... c) some useful stubs c.1) luapi_open( lua_State ** , stacksize, alertdisplay ) receives a pointer to a lua_State pointer, and - checks whether it's null ( support for redundant opens ) if it's, opens a new interpreter, opens all libraries, and returns the pointer if a non null alertdisplay is passed, Alert_Stub is set to this one. lua_State my_Lua; my_Lua = luapi_open( & my_Lua, 0, ... ) c.2) alert function every interpreter created using luapi_open( ) receives as the alert function a static function, which in turns - if a stub function is defined, pass the message to it else puts the message on stderr - increments a count of alert calls ( a module static int ) c.3) luapi_Alert_Stub the function Alert_Stub sets the message displayer to any void function you may want, without requiring it to know and manage the lua protocol ( it receives the bare message ) This function returns the current Stub ... c.4) luapi_alerted when Lua calls the alert function, may be the end user sees the message, but your code does not. luapi_alerted returns the current count of alert calls, and resets it to zero. it's good advice to call alerted at the end of every critical lua function ( dofile, call, etc ) to see if something wrong happened and to take appropiate actions ... d) one useful thing for Lua 3.x host porting ... luapi_Lua is a lua_State automatically opened if luapi_open( NULL, ... ) is invoked, and used ( thru luapi_check ) if most of the luapi functions are invoked with lua_State NULL ... What it does not Any userdata management ... please, see below BUGS AND IMPROVEMENTS All are welcome .. mnicolet@satlink.com mnicolet@speedy.com.ar A word regarding userdata I have no experience with userdata, and ( I confess ) it's not easy for me to *fully* understand it, and I cannot devise a simple test to 'taste' and learn about. It seems a lack. I would like, by example, to 'synchronize' some host variables with the ones seen by the interpreter, without resorting to every time set them as simple Lua globals or table members, and because it would be nice that some lua functions could write to them, without use of a C function. I suspect userdata would be the answer, but cannot figure exactly how. If someone can advice or send me a snippet ... Because of the above, luapi does nothing useful regarding userdata, and under certain circumstances ( where it does not 'know what's doing' ) it invokes lua_error ... At least, I think luaval_t must have another member, to accomodate the tag ... is't ok ?