Re: Proposal: Virtualization of Filesystem Operations for Lua 5.2.

[Georg Lehner <jorge-lua@...>]

Hello!

Thanks for the interest and for all the inputs to the topic.

One thing I like about Lua is, that its simple and small.

The "fs" proposal tries to closely follow this principle. It's scope is 
narrow:

- read/write contents of static files
- manipulate and query filesystem trees

This is enough to write a wealth of day-to-day utilities.  For the 
remaining part of
the universe other solutions will do.

In my opinion, only the lowest common denominator of all (!) filesystems 
should be
covered by a Lua core "fs", and this would be: (directory) trees with 
sets of named
bytestreamcontainers (files) in each nodes.  Practical extensions:

- directed graphs instead of trees
- attributes for filesystem objects

While the unification of stream and block io in the unix world is a 
great thing, I
tend to not make it part of the "fs" concept.  stdin/stdout/stderr
is a concept which might be available and supported on platforms of 
typical Lua
standalone interpreters, but is not a feature found and needed in all 
situation
where a Lua interpreter is embedded.   Let the Lua embedding software do the
respective work if required by the application.  Also: there is still "io"
around to process stdin/stdout


Some comments on the following two acpects which have been mentioned in 
the discussions:

UTF8 (or whatever other character set and format of) filenames:

"fs" is thought to be a namespace, where each path element is a Lua 
string (whatever a
Lua string might be in some Lua incarnation) and the '/' character is 
the separator
between paths and the symbol for the root of the namespace tree.
Every subtree below a 'mount point' is handed over to the filesystem 
driver responsible
for this exact 'mount point' (e.g. LuaFilesystem).  The filesystem 
drivers converts "fs"
subtrees to-and-fro the underlying filesystem naming convention.


Asynchronouos I/O:

This is a challenging topic and I would like to see a good proposal how 
to make it fit in
nicely with the Lua core.

However I have the impression, that there is no standard way for 
concurrency in Lua except
coroutines.  Adding "fileevents" or asynchronous I/O to the Lua core 
would require to add
a central dispatching mechanism to it (event loop, wait queues, or the 
like) and this seems
to much additional complexity and intrusion with respect to the scope of 
the "fs" proposal.

- - -
Both issues, however can be treated nicely *with* "fs", even if not *in* 
"fs":

If a program wants to operate on file:

local path = "/this/is/a/lengthy/path/with/a/mountpount/and/file/below"

and e.g. the host root directory is mounted via a filesystem driver (io, 
os and LuaFilesystem)
at "mountpoint" it will

  local fh = fs.open(path)

which will internally assign

  fh.fd = io.open("/and/file/below")

which stores the returned "io" filehandle for later use.  This later use 
of need not necessary
be a "fs" operation, but could for example be lfs.lock(fh.fd,"r"), or 
asynchronous access with
another Lua library written for this purpose.

 local path,fsdriver = fs.volpath(path)

returns "/and/file/below" in path, and the filesystem driver in 
fsdriver, so that an application
knows who is responsible for dealing with "/and/file/below".  On the 
other hand fh.volume (from
the fs.open(path) call) is the same as fsdriver.  An application can 
thus adjust operation on
a specific file or filehandle of "fs".
- - -

Best Regards,

    Jorge-León