Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

[Dimiter 'malkia' Stanev <malkia@...>]

What I would like to see is way to override every "C" function.

Instead of lua calling directly the "C" runtime function, it can be 
called through a "C" struct table with pointers.

Later each can be overriden. This would increase the embeddability, 
especially if one is provided with DLL.

The table can be static (global), but it could be allocated per "L" 
state, and initialized to default to the used "C" runtime calls.

-------

Alternatively, for one pet project of mine I've started with using the 
preprocesor to hook them, but later gave up - i simply did not need this 
for the pet project, but if it later had to be used in a tool or app I 
would've come back at it:

Something like this:

//luabox.c

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

#ifndef LUA_API
#define LUA_API
#endif

LUA_API FILE*  luabox_stdin;
LUA_API FILE*  luabox_stderr;
LUA_API FILE*  luabox_stdout;
LUA_API void   luabox_fprintf(){}
LUA_API void   luabox_fclose(){}
LUA_API void   luabox_fread(){}
LUA_API void   luabox_feof(){}
LUA_API void   luabox_fscanf(){}
LUA_API void   luabox_fgets(){}
LUA_API void   luabox_fputs(){}
LUA_API void   luabox_fopen(){}
LUA_API void   luabox_ferror(){}
LUA_API int    luabox_fputc( int c, FILE *stream ){return 0;}
LUA_API int    luabox_fflush( FILE *stream ){return 0;}
LUA_API size_t luabox_fwrite( const void *buffer, size_t size, size_t 
count, FILE *stream ){return 0;}
LUA_API void   luabox_getenv(){}
LUA_API void   luabox_getc(){}
LUA_API void   luabox_ungetc(){}
LUA_API void   luabox_clearerr(){}
LUA_API void   luabox_clock(){}
LUA_API void   luabox_exit(){}
LUA_API void   luabox_setvbuf(){}
LUA_API void   luabox_putchar(){}
LUA_API void   luabox_remove(){}
LUA_API void   luabox_rename(){}
LUA_API void   luabox_system(){}
LUA_API void   luabox_tmpnam(){}
LUA_API void   luabox_tmpfile(){}

#undef  stdin
#undef  stdout
#undef  stderr

#define stdin    luabox_stdin
#define stdout   luabox_stdout
#define stderr   luabox_stderr
#define fopen    luabox_fopen
#define fputc    luabox_fputc
#define fprintf  luabox_fprintf
#define fscanf   luabox_fscanf
#define fwrite   luabox_fwrite
#define fread    luabox_fread
#define fwrite   luabox_fwrite
#define feof     luabox_feof
#define fgets    luabox_fgets
#define fputs    luabox_fputs
#define ferror   luabox_ferror
#define fflush   luabox_fflush
#define fclose   luabox_fclose
#define getenv   luabox_getenv
#define getc     luabox_getc
#define ungetc   luabox_ungetc
#define clearerr luabox_clearerr
#define clock    luabox_clock
#define exit     luabox_exit
#define setvbuf  luabox_setvbuf
#define putchar  luabox_putchar
#define remove   luabox_remove
#define rename   luabox_rename
#define system   luabox_system
#define tmpfile  luabox_tmpfile
#define tmpnam   luabox_tmpnam


On 11/2/2011 12:53 PM, Georg Lehner wrote:
> Hello!
>
> I propose to include a virtualization layer for filesystem operations in
> Lua 5.2.
>
> Problems with the current approach to filesystem operations are:
>
> - incomplete functionality (e.g. directory operations)
> - their behaviour depends on the platform where Lua is built
> - in some environments (deeply embedded..) the current functionality
> might be
> meaningless (and must be stripped out when porting)
>
> Advantages of virtualization, besides releaving the problems, are (see
> also [1])
>
> - improved portability both for the Lua interpreter and for Lua scripts
> - extensibility, generalization and reusability: Write portable Lua scripts
> for anything what can be made to look like a filesystem.
>
> Tcl/Tk features a successful filesystem virtualization layer[2].
>
> A simplified proof of concept implementation is available at [1] with
> about 400 lines
> of Lua code. I claim that an implementation of the virtualization layer
> will be
> slightly less code then actually contained in the Lua interpreter
> (without including
> a "real" filesystem driver).
>
>
> I propose the following features (in more or less erratic order):
>
> Extension of existing functions
>
> - open(path,"d+") creates a directory
> - open(path,"d"[,attributes]) returns an iterator over the directory path,
> returning filenames of existing files matching the attributes table
> - remove(path) also removes (empty) directories
>
>
> Additional functionality
>
> - stat(path,[attributes]) sets/gets file attributes of path
>
> - cd([path]) sets/gets the working directory of the current function
> - mount(volume, path[,options..]) attaches the volume (see later) onto
> the mount point path
> - bind(from, to [options..]) creates links between directory trees
> - unmount(volume[, path]) deattaches volumes or links from the mount point
>
> See the paper "Lexical File Names in Plan 9 or Getting Dot-Dot Right" from
> Rob Pike [3] for the proposed mount/bind/unmount semantic
>
> Implementation specifics
>
> - each function has its own filesystem, which is by default inherited from
> its creator, but can be restricted (for sandboxes)
> - '/' is the path separator, each filesystem driver provides translation to
> its specific path representation
>
> - filesystem drivers implement only low level access to filesystem objects:
> read/write strings from/to files, create/delete files and directories,
> query directories and file attributes.
> - the virtualization layer implements higher level features like lines(),
> formatted read and write.
> - Lua scripts and pure Lua libraries implement high level applications like
> file copying or removing.
>
> - A lua state has to be set up by its creator (e.g. the standalone Lua
> interpreter) to mount a specific filesystem driver. This is however an
> optional step
> - The current functionality of the io and os library can be maintained in
> the Lua interpreter and expanded as a filesystem driver which is mounted
> by default. This maintains backwards compatiblity for existing Lua script
> without to much hassle and overhead.
>
> - Volume instance specific identifiers like "C:", "D:",
> "http://www.magma.com.ni/sw/lua/luafs"; are set up as the "root" of a
> volume by the file system driver.
>
>
> Permissions and Authorization
>
> - Authorization can be done when a volume is instantiated, i.e. not Lua
> interpreter specific but filesystem driver specific
> - Lua only needs to operate on already accessible filesystem objects,
> permission
> manipulation can be filessystem driver specific
>
>
> Terms
>
> - filesystem object: any object which can be represented by a directed
> graph
> with named edges and where each node can hold a set of named objects with
> a byte stream content and optional attributes
> - filesystem driver: Lua library which operates on a specific class of
> filesystem objects
> - volume: filesystem driver instance which operates on a filesystem object.
>
>
> Filesystem class examples
>
> - Lua tables
> - Windows Registry
> - Websites
> - IMAP Mailaccounts
> - Floppies, USB-Sticks, lokal and remote Windows drives
> - zipfiles, tarballs, any other archives
> - source repositories, git, svn, ...
> - C-String arrays/data structures for deeply embedded Lua
>
>
> The case of the Lua module search path
>
> - with the proposed union mounts, search paths for Lua modules
> might become obsoleted by binding all locations where required
> modules live to a standard path in the virtual filesystems namespace
>
> Best Regards,
>
> Jorge-León
>
>
> [1] http://sw.magma.com.ni/LuaFs
> [2] http://wiki.tcl.tk/2138
> [3] http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames