 |
lua-l archive |
|
|
||
|
On 5 Nov 2015 01:56, "Sean Conner" <sean@conman.org> wrote:
>
> It was thus said that the Great Stefano once stated:
> > On 4 November 2015 at 20:00, Sean Conner <sean@conman.org> wrote:
> > >
> > > While I could store both luuid's "uuid" and uuid's "uuid" modules in the
> > > same LEM, I don't have a way to specify which one to use. That *is*
> > > something I would need to work on. But as it stands, the LEM format can
> > > easily handle different operating systems, different architectures and
> > > different Lua versions (look closely---you'll see I have an LPeg 0.12 module
> > > for Lua 5.1 and for Lua 5.2).
> >
> > That does not solve the issues I raised:
> > What about file conflicts across different packages (LEMs)?
>
> The original intent of my project was to distribute and run a Lua
> application directly from a LEM file. That proved to be ... optimistic at
> best [1] (I was able to run LuaRocks directly from a LEM file, but LuaRocks
> is pure Lua, and I had to hack into io.open(), dofile() and loadfile()).
> But assuming I drop that requirement and instead just unpack everything onto
> disk.
>
> > How to perform updates when one package nest another one?
> > How to keep the complexity manageable while still achieving
> > rollback-on-failure safety?
> > Ecc ecc
>
> You face those same problems. Lua doesn't handle module versions at all
> (I don't mean between Lua versions, I mean version 1.3 and 1.4 of "foobar")
> and if there's a fundamental incompatability, the best you can do is "I'm
> sorry, Dave. I can't let you do that."
Lua doesn't but ULua does.
Semantic versioning allows developer to tell when there is a breaking change.
And incompatible versions can coexist and be loaded at the same time.
Also, I do not face those same problems.
Indeed my code, publicly available, deals with all the mentioned issues in a trivial way (of course, also thanks to the design choices which you don't seem to like).
>
> But aside from that---my system had operating system, architecture, Lua
> version (minimum and maximum the module could run under) and module version.
> So, running through ULua (I downloaded it from
> http://www.scilua.org/ulua.html) as a LEM module (hypothetically speaking
> here), here's how I would handle things. First, a listing of the LEM (not
> everything is included as I didn't want to spend too much time trying to
> track down what everything does, but this should get the point across):
>
> Linux x86 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
> Linux x86 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
> Linux x86-64 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
> Linux x86-64 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
> Darwin x86 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
> Darwin x86 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
> Darwin x86-64 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
> Darwin x64-64 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
> Windows x86 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
> Windows x86 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
> Windows x86-64 MIT/X11 Lua 5.1 5.1 1.6.3-103 MODULES/lfs
> Windows x86-64 MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/curl
> MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL
> MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL.safe
> MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL.utils
> MIT/X11 Lua 5.1 5.1 0.3.1 MODULES/cURL.impl.cURL
> MIT/X11 Lua 5.1 5.1 1.0-beta6 MODULES/pkg.init
> MIT/X11 1.6.3-103 DOCS/lfs/doc.css
> MIT/X11 1.6.3-103 DOCS/lfs/license.html
> MIT/X11 1.6.3-103 DOCS/lfs/index.html
> MIT/X11 1.6.3-103 DOCS/lfs/examples.html
> MIT/X11 1.6.3-103 DOCS/lfs/luafilesystem.png
> MIT/X11 1.6.3-103 DOCS/lfs/manual.html
> MIT/X11 1.0-beta6 DOCS/pkg.init/README.md
> MIT/X11 1.0-beta6 DOCS/pkg.init/LICENSE
> MIT/X11 1.6.3-103 TESTS/lfs/test.lua
> MIT/X11 BIN/upkg
> MIT/X11 BIN/upkg.cmd
> Linux x86 BIN/zip
> Linux x86 BIN/unzip
> Windows x86 BIN/zip
> Windows x86 BIN/unzip
> Windows x86 BIN/chmod
> Linux x86 MIT/X11 7.42.1 LIB/libcurl
> Linux x86-64 MIT/X11 7.42.1 LIB/libcurl
> Windows x86 LIB/libintl3
> Windows x86 LIB/libiconv2
> Windows x86 MIT/X11 7.42.1 LIB/libcurl
> Windows x86-64 MIT/X11 7.42.1 LIB/libcurl
> FILE/upkg
>
> As I'm running Linux x86, I can ignore anything for Linux x86-64, Darwin
> (how OS-X identifies itself) and Windows. Files without an operating
> system/architecture designation can be used on anything.
>
> First up, lfs. I already have lfs version 1.6.3 installed on my system,
> so I can skip that module. I do *not* have curl currently installed, so I
> can copy that module out (keeping track of versions like LuaRocks does)
> somewhere into my Lua CPATH (let's make that a configuration option of LEM).
> I don't currently include dependency information (but I'll skip that for
> now). I can extract the module and as a safety check, I can run 'ldd' over
> it to make sure I can run it:
>
> [spc]lucy:/tmp/foo/ulua>ldd ./lcurl/0_3_1+103/Linux/x86/-lcurl.so
> ./lcurl/0_3_1+103/Linux/x86/-lcurl.so: /lib/tls/libc.so.6: version LIBC_2.4' not found (required by ./lcurl/0_3_1+103/Linux/x86/-lcurl.so)
> libcurl.so.4 => not found
> libc.so.6 => /lib/tls/libc.so.6 (0x00111000)
> /lib/ld-linux.so.2 (0x00b76000)
>
> Failure, but I haven't gotten to installing it yet, so nothing damaged
> (yes, I am running an older Linux distribution, no I have no plans of
> upgrading any time soon as it works---the last time I upgraded a system
> around here it was ... painful [2]). And even if that succeeded, I don't
> have libcurl4. But I think that's why you include the library, so let's
> carry on as if nothing happened. The Lua based cURL modules can be copied
> appropriately, as well as pkg.init. I'll skip over the DOCS and TESTS
> "directories" for now.
>
> "upkg" and "upkg.cmd" don't exist in my path, so we can copy those to a
> directory in my path that I control (most likely $HOME/bin) [3]. Having the
> upkg.cmd won't hurt as I won't be using it.
>
> zip and unzip exist in my path, so we can skip those. Ah, there's
> libcurl we could install. And like the curl module, we can pretest this
> thing:
>
> [spc]lucy:/tmp/foo/ulua>ldd ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so
> ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so: /lib/tls/libc.so.6: version LIBC_2.11' not found (required by ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so)
> ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so: /lib/tls/libc.so.6: version LIBC_2.7' not found (required by ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so)
> ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so: /lib/tls/libc.so.6: version LIBC_2.4' not found (required by ./clib_libcurl/7_42_1+3/Linux/x86/libcurl.so)
> librt.so.1 => /lib/tls/librt.so.1 (0x002c8000)
> libc.so.6 => /lib/tls/libc.so.6 (0x006f9000)
> libpthread.so.0 => /lib/tls/libpthread.so.0 (0x008fa000)
> /lib/ld-linux.so.2 (0x00b76000)
>
> Hmm ... failure (I did mention I was running an older system here, and I'm
> probably not your target audience for ULua). That's the only dependency we
> can't fulfill (at least, given that I don't have curl 0.3.1 installed).
> Backing out is fine---we can remove curl 0.3.1 as I never had it installed
> to begin with. Same with pkg.init. I can delete ukpg and ukpg.cmd since
> they were just installed.
I am already compiling for an old libc, but going further in the past is not a realistic option for me I am afraid.
>
> But assuming I already had cURL installed (current version is 0.3.2 by the
> way). That should be okay, according to semversion, so I could have skipped
> MODULES/curl and (had I the dependency information) skip libcurl as well.
> Then pkg.init and upkg would be installed and everything would be fine.
>
> Other things to keep in mind---what if I installed a module prior to using
> ULua? I have modules installed from source, as well as several installed
> via LuaRocks. That's something to keep in mind.
Not an issue.
Either you move them to the ulua folder or append the appropriate paths to package.lua and package.clua.
A minimum---try including
> a module prior to installing it to see if I might already have one
> installed. If so, and I can't determine the version (because it lacks a
> _VERSION or VERSION or _version or version field) then I can either error
> out or ask the user---something along those lines.
ULua can use both its own packages and the ones installed by other means (see above).
>
> How to handle the case where I might be installing a module that breaks a
> previously installed module? Error out and inform the user. Maybe offer to
> install everything in a sandboxed location. If everything has always been
> installed via LEMs (hypothetically speaking) perhaps attempt to do a "fake"
> upgrade first to see if it's possible not to break anything and either say
> "yes, you can upgrade and not break anything" or "sorry, we can't upgrade
> because foo will break."
>
> Keeping this all in mind, I don't see where the issue of nested packages.
There are.
For instance I rely on dir move because otherwise you cannot remove loaded binaries (and this is always the case for lfs for instance) as part of an update.
In any case, my philosophy is to offer something that just work.
Conflicts have to be checked by myself, having it error out on the user is not an option.
That said, yes, with non trivial effort one can accommodate nested modules.
I am just stressing it is not easy as you are suggesting it is and, my time being limited, I have more important features / projects to work on.
As I said before, I welcome code contributions.
>
> > > I propose to you---what would you do if someone would like to use package
> > > manager to install json4lua? Or a different example---someone needs
> > > numerical integration and thus, would like to use math-rungekutta but can't,
> > > because it "conflicts" with math-evol.
> >
> > The thing is, right now math-evol is taking over math, which is
> > clearly bad (we already discussed about that).
> > Again, a problem that would be solved by having project name = module name.
> > Then, hopefully, something different from math would have been used.
>
> Technically, it's taking over 'Math', and both modules are written by the
> same person and are released under a common namespace module (hmmm ...
> sounds familiar), named because it's a port of similarly named Perl modules
> (Math::RangeKutta, Math::Evol, etc.). The author isn't doing anything wrong
> here.
As I stated before, I introduce *additional* constraints (just no nesting really).
These gives you a number of benefits which, I am strongly convinced, outnumber the minor inconveniences.
>
> > > Final parting words---if "conflicting modules" is too complex to handle,
> > > then please, remove "org" from your project.
> >
> > I welcome your ides on how to handle it.
> > Not in the sense of storing them in a package. I am talking about the
> > package management logic here.
>
> Treat the module names as an opaque string and stop trying to shoehorn
> everything into what appears to be a top-level module. Lua does this
> internally (after require "org.conman.table", require "pl", require
> "socket"):
>
> > show(package.loaded)
> string table: 0x9acb618
> org.conman.table table: 0x9add188
> package table: 0x9acba50
> _G table: 0x9aca450
> os table: 0x9acd3c8
> table table: 0x9acb100
> math table: 0x9acdf90
> coroutine table: 0x9acb718
> pl.utils table: 0x9ae3328
> debug table: 0x9ace990
> pl true
> io table: 0x9accbc0
> socket.core table: 0x9aeea48
> pl.import_into function: 0x9ae29b0
> socket table: 0x9aeea48
> org.conman.net table: 0x9ae3a28
> pl.compat table: 0x9ae9718
>
> Just because "org.conman.table" is stored on disk as
> "org/conman/table.lua" doesn't necessarily mean "org" is the top level
> module here. It's "org.conman.table". Yes, this might complicate your
> package manager some, but I was able to deal with a ZIP file with duplicate
> filenames!
The point of contention is that 'some'.
If it is so trivial, you would really do me a great favor by contributing a patch that solve this contentious point once and for all.
>
> > When I get around to implement rocks aggregation (meaning all your
> > rocks will end up under a single org package), you'll be free to opt
> > out if you want to.
>
> ORG IS NOT THE NAME OF MY MODULES! HOW MANY TIMES DO I HAVE TO SHOUT THAT
> BEFORE YOU UNDERSTAND?!
At this point, I wonder if you're being sarcastic.
>
> -spc (Wait a second! Can't LuaRocks handle binary distributions as well?)
>
> [1] While I could load Lua modules written in Lua directly from the ZIP
> file, those written in C could not. It was a limitation of dlopen()
> (the Unix API to load a shared object) and it wouldn't surprise me
> if there wasn't a similar situation under Windows. And while I
> could hook onto the Lua functions that open a file for reading, I
> never did get a clear idea of how to handle opening a file for
> writing.
>
> [2] http://boston.conman.org/2015/10/01-18
>
> [3] upkg.cmd *could* probably have a Windows operating system associated
> with it. "upkg" itself is for Unix. I suppose I could include a
> "unix" operating system label for such a condition.
>