Re: Lua library bank? (Was: Ruby philosophy vs Lua philosophy

[Andrew Starks <andrew.starks@...>]

On Fri, Mar 8, 2013 at 3:36 PM, Pierre-Yves Gérardy <pygy79@gmail.com> wrote:

>
> Hosting all packages on GitHub does not give you a star rating system,
> but you can see how popular a given package is, how reactive the
> author is to bug reports, etc..
>
> -- Pierre-Yves

I like the idea of using GitHub as the base.

Also, there isn't much wrong with luarocks that couldn't be fixed or
extended away. For me, I have trouble with it in Windows and I can't
seem to make it believe me when I ask it to use other directories. The
second problem is probably me, and the first might be better fixed
with a another package.searcher.

Wants/desires/I-haven't-figured-it-out-yet are:

- Version/Updating modules.
- Keeping module repositories and the installed builds of them
straight, so my patches are more easily managed.
- Modules that need non-lua libraries that I don't have installed and
don't really want to install in my /usr/local/ tree.
- Modules that need hard-to-build libraries (like ZMQ on Windows).
- Ability to change environments for using, developing, testing, and deployment.

Some ideas...

I think Lua's method for storing .o/.so/DLLs and lua scripts makes
sense for the core language. It feels like an approach for package
management that is aiming for simplicity might be better off
considering ways to make cross-platform deployment "not suck." Perhaps
the addition of a package searcher and an aping of TeX/Context/LaTeX
TEXMF trees? Instead of separating  cpath and path directories into
different roots (or at least roots at different levels), use a single
tree, such as this one that i just thought up right now:
moddir/repos      --(optional, probably only in the system tree) a git
repo that only has module URLs...
                            --I guess the way that luarocks does it
now works fine. No need to reinvent that...
moddir/git            --the raw repos for all installed modules.
"moddir/git/mymodule", etc
moddir/lua           --instead of /usr/local/share/lua/5.2
moddir/lib            --instead of /usr/local/lib
                            --perhaps the makefile could automate
putting not-found, dependent, non-lua libraries here too, such as
"curl" or "expat"?
moddir/resources  --for other non-exec binaries that a module or
distribution might need. Would help with more application-y
distribution, like Love
moddir/doc      --perhaps typing luadoc mymodulename looks in all of
the moddir trees for that module's documentation and loads a viewer?
                        -- Again, stolen from TeX: texdoc
moddir/include --perhaps for header files that are used for FFI or C
libraries that are being built with lua?
moddir/bin        --perhaps for lua-centric commands, like build
utilities? Would help with Love?

Standard array of 'moddir's with fall-through rules apply. The hope is
that it works well on Windows and *nix. On *nix machines, the
system-level moddir goes in '/usr/local/share/lua/5.2/luamod', Windows
in '/Windows/System32/lua/5.2/luamod'. Then, also, the user would have
one installed at "~/lua/5.2/luamod" and "$HOMEPROFILE/Documents" or
whatever it is in Windows. :)

I'm positive that I'm violating 75% of everyone's sensibilities.
Precedent has been set for this approach, by TeX, which is a battle
hardened world. Unlike with Lua, I can make a wicked complicated,
extremely dependency-laden LaTeX project compile on Windows, Mac and
Linux, all from very different distributions (MacTeX, MikTeX and
TexLive). They even work well between WinEdt and Sublime and our CI
server.

To help solve versioning issues, swapping environments and keeping
repos and module directories straight:

- keep the whole repo in tact in 'moddir/git'
- use git fetch as the caching mechanism for detecting updates.
- build in the repo's build directory (moddir/git/mymodule/build).
    -- The module author can choose to .gitignore in there or not.
    -- If not, they can have subdirectories for platform binaries in
"bin" and in "libs". This would be good for Windows.
    -- If repo bloat is a problem, use branches to solve it. Have a
"with-binaries" branch that includes the binary files.
    -- The repo's build directory has the same tree structure as a
regular moddir.
- Maybe most important: symlink all of the files in the build
directory (moddir/git/mymodule/build/lua, etc) to that moddir tree's
root versions (moddir/lua, moddir/lib, etc)

The hope is that...

- when that module is updated and rebuilt, the symlinks are updated.
- if the user wants to patch the module, they can branch it and the
symlinks still work.
  -- if we enforce the policy that they must branch, luarocks can
still maintain updates from the author, without breaking local
changes.
  -- If luarocks detects local commits on master, then it can complain
and offer a "reset --hard" alias.
- luarocks can use branching to provide facilities for named "module
environments" that can be envoked for testing, development, general
use, etc.

Environment switching could be done the normal way (like a script that
holds the environment names and sates), or it could be done with git
subtree merging. With subtree merging, the whole 'moddir/git' is a git
repo and "git subtree add ..." is used to add each module. Each module
becomes a branch, with only it inside of it. They would be named
something like "mymodule_module". New environment branches can be made
from master. In those branches, modules could be updated/deleted/etc.
These branches might be named "debug_myproject_env", etc.

Shipped binaries might be organized like so (stealing from TeX):

moddir/git/build/lib/common --default.
moddir/git/build/OSX/10.6 --perhaps if you're 10.7, it would use this?
moddir/git/build/OSX/10.5
moddir/git/build/OSX --default for mac
moddir/git/build/Windows --etc...

only relevant binaries are symlinked into 'moddir/lib'

I've gone on long enough. Sorry. You are all saints.

Andrew Starks
Tightrope Media Systems

"Your stupid." -favorite bumper sticker