lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


I propose to include a virtualization layer for filesystem operations in Lua 5.2.

Problems with the current approach to filesystem operations are:

- incomplete functionality (e.g. directory operations)
- their behaviour depends on the platform where Lua is built
- in some environments (deeply embedded..) the current functionality might be
  meaningless (and must be stripped out when porting)

Advantages of virtualization, besides releaving the problems, are (see also [1])

 - improved portability both for the Lua interpreter and for Lua scripts
- extensibility, generalization and reusability: Write portable Lua scripts
   for anything what can be made to look like a filesystem.

Tcl/Tk features a successful filesystem virtualization layer[2].

A simplified proof of concept implementation is available at [1] with about 400 lines of Lua code. I claim that an implementation of the virtualization layer will be slightly less code then actually contained in the Lua interpreter (without including
a "real" filesystem driver).

I propose the following features (in more or less erratic order):

Extension of existing functions

- open(path,"d+") creates a directory
- open(path,"d"[,attributes]) returns an iterator over the directory path,
  returning filenames of existing files matching the attributes table
- remove(path) also removes (empty) directories

Additional functionality

- stat(path,[attributes]) sets/gets file attributes of path

- cd([path]) sets/gets the working directory of the current function
- mount(volume, path[,options..]) attaches the volume (see later) onto
  the mount point path
- bind(from, to [options..]) creates links between directory trees
- unmount(volume[, path]) deattaches volumes or links from the mount point

See the paper "Lexical File Names in Plan 9 or Getting Dot-Dot Right" from
Rob Pike [3] for the proposed mount/bind/unmount semantic

Implementation specifics

- each function has its own filesystem, which is by default inherited from
  its creator, but can be restricted (for sandboxes)
- '/' is the path separator, each filesystem driver provides translation to
  its specific path representation

- filesystem drivers implement only low level access to filesystem objects:
  read/write strings from/to files, create/delete files and directories,
  query directories and file attributes.
- the virtualization layer implements higher level features like lines(),
  formatted read and write.
- Lua scripts and pure Lua libraries implement high level applications like
  file copying or removing.

- A lua state has to be set up by its creator (e.g. the standalone Lua
  interpreter) to mount a specific filesystem driver.  This is however an
  optional step
- The current functionality of the io and os library can be maintained in
  the Lua interpreter and expanded as a filesystem driver which is mounted
by default. This maintains backwards compatiblity for existing Lua script
  without to much hassle and overhead.

- Volume instance specific identifiers like "C:", "D:",
  ""; are set up as the "root" of a
  volume by the file system driver.

Permissions and Authorization

- Authorization can be done when a volume is instantiated, i.e. not Lua
  interpreter specific but filesystem driver specific
- Lua only needs to operate on already accessible filesystem objects, permission
  manipulation can be filessystem driver specific


- filesystem object: any object which can be represented by a directed graph
with named edges and where each node can hold a set of named objects with
  a byte stream content and optional attributes
- filesystem driver: Lua library which operates on a specific class of
  filesystem objects
- volume: filesystem driver instance which operates on a filesystem object.

Filesystem class examples

- Lua tables
- Windows Registry
- Websites
- IMAP Mailaccounts
- Floppies, USB-Sticks, lokal and remote Windows drives
- zipfiles, tarballs, any other archives
- source repositories, git, svn, ...
- C-String arrays/data structures for deeply embedded Lua

The case of the Lua module search path

- with the proposed union mounts, search paths for Lua modules
  might become obsoleted by binding all locations where required
  modules live to a standard path in the virtual filesystems namespace

Best Regards,