Re: [proposal] require resources

["Soni \"They/Them\" L." <fakedme@...>]

On 2020-01-20 1:10 a.m., Sean Conner wrote:
> It was thus said that the Great Soni They/Them L. once stated:
> > If you had:
> > 
> > LUA_PATH/foo/init.lua
> > LUA_PATH/foo/bar/init.lua
> > LUA_PATH/foo/bar/res.txt
> > 
> > then require("foo.bar", "res.txt") would get you the contents of 
> > LUA_PATH/foo/bar/res.txt
>
>    Here's my current package.path, broken down to it's easy to see the
> contents:
>
> 	/home/spc/.luarocks/share/lua/5.3/?.lua
> 	/home/spc/.luarocks/share/lua/5.3/?/init.lua
> 	/home/spc/.luarocks/lib/lua/5.3/?.lua
> 	/home/spc/.luarocks/lib/lua/5.3/?/init.lua
> 	/usr/local/share/lua/5.3/?.lua
> 	/usr/local/share/lua/5.3/?/init.lua
> 	/usr/local/lib/lua/5.3/?.lua
> 	/usr/local/lib/lua/5.3/?/init.lua
> 	./?.lua
> 	./?/init.lua
>
>    And to make things even worse, let's say I have an app:
>
> 	org.conman.app.gopherclient
>
>    As currently defined, when I do
>
> 	app = require "org.conman.app.gopherclient"
>
> Lua will check for the following:
>
> /home/spc/.luarocks/share/lua/5.3/org/conman/app/gopherclient.lua
> /home/spc/.luarocks/share/lua/5.3/org/conman/app/gopherclient/init.lua
> /home/spc/.luarocks/lib/lua/5.3/org/conman/app/gopherclient.lua
> /home/spc/.luarocks/lib/lua/5.3/org/conman/app/gopherclient/init.lua
> /usr/local/share/lua/5.3/org/conman/app/gopherclient.lua
> /usr/local/share/lua/5.3/org/conman/app/gopherclient/init.lua
> /usr/local/lib/lua/5.3/org/conman/app/gopherclient.lua
> /usr/local/lib/lua/5.3/org/conman/app/gopherclient/init.lua
> ./org/conman/app/gopherclient.lua
> ./org/conman/app/gopherclient/init.lua
>
>    Right now, Lua goes through the package.searchers array, calling each
> function in turn.  The second searcher will break the value in package.path
> by the template splitting character (default of ';'---not shown in the list
> above) and replace any dots in the module name with the systems directory
> separator ('\' in Windows, '/' in non-Windows, but it could technically be
> anything).  Then for each template in package.path, it substitues the
> substitution point (defaults to '?') with the modified module name.
>
>    I don't think this is anything you don't know, but look at what is
> *actually* happening here---Lua is constructing special values it gives to
> fopen() (the C function responsible for opening a file)---Lua itself has no
> concept of "directories".  It doesn't have the concept of extensions
> (otherwise, why include them in the template?).  It just constructs special
> opaque (to Lua) values and hands them off to fopen().  That's it.
>
>    How would you fit "gopher.icon" into this?  Remember, there is no concept
> of a directory or even extensions in Lua.  It's all must magic opaque values
> as far as Lua is concerned.  Given the line:
>
> 	gc,icon = require("org.conman.app.gopherclient","gopher.icon")
>
> where would "gopher.icon" even live?  At the same level as
> org.conman.app.gopherclient.lua?  What if gopherclient was in fact a
> directory?  Or both org/conman/app/gopherclient.lua and
> org/conman/app/gopherclient/? (both a file *and* a directory?) I do that
> quite often for my own modules and it works fine.
>
>    And is it just one resource?  What if there are more?  Like a database of
> text translations for different langauges?  Multiple icons for different
> windows?  Sounds?
>
>    It might seem like a simple request, but it's not.

The same can be said of require.

Again it would just be replacing patterns:

icon = require("org.conman.app.gopherclient","gopher.icon")

replace the left's .'s with /'s, add / at the end, add right at the end.

if you had resource path /usr/lib/lua/5.4/? for example, it would try to 
open /usr/lib/lua/5.4/org/conman/app/gopherclient/gopher.icon

> > HOWEVER this would only apply to the default loaders. what I mean by 
> > that is that one could make a zip file loader and then you can have your 
> > whole module (and submodules) as a zip file instead.
> > 
> > foo.zip
> > | init.lua
> > | bar/init.lua
> > | bar/res.txt
> > 
> > which is a lot harder than opening a file in LUA_PATH.
>
>    Not really.  Back in 2014 I did some experiments with doing just this (for
> Lua 5.1) [1].  I got to the point where I could run LuaRocks entirely out of
> a ZIP file.  I even patched loadfile() and dofile() to handle opening of
> files in the ZIP, but they only supported read-only files.  I could not get
> Lua modules written in C working, but that's due to a limitation of loading
> dynamic objects (only from disk, not from memory [2]).
>
>    All it requried (for Lua modules in Lua) was adding a new entry into the
> package.searchers array.  And why would a require() with resources just be
> limited to the included searchers?

It wouldn't. Add a resource searcher/loader.

>    -spc
>
> [1]	Some of which can be viewed here:
>
> 		https://github.com/spc476/LEM
>
> [2]	Don't ask me why not, ask the guys who wrote the OS why the
> 	limitation.  I'm not reponsible for that one.

Write your own dynamic loader/linker ;)

>