paving cowpaths with libraries (was Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available))

[Jay Carlson <nop@...>]

On Feb 13, 2012, at 1:45 PM, Axel Kittenberger wrote:

>> I want to cut features ruthlessly until we mostly agree we all use something from time to time, and it looks and acts mostly the same
> 
> That would be vanilla Lua as you get it. I don't get it what you are
> aiming at, or just don't see through it through the badly written
> text.
> 

I'm sorry if I was unclear. I either need more coffee or less coffee. 

My thesis is that there is an ad-hoc basic library everybody already uses. It's not solely what's in _G (and when *was* the last time you called string.reverse?)

I think the reason that some previous standard library efforts lost momentum was that they did too much new stuff, or attempted to do things that there was not general agreement on. I don't want to see another object system, or even multiple object systems. I think they reduce the likelihood any library as a whole is widely adopted--even though you don't have to use them, they are irritants. This goes double for the use of an object system to implement a library.

The jargon "paving the cowpaths" describes my goal; these are things I think almost everybody does from time to time and usually in the same way. (I have a slight bias for inclusion of things not as commonly used but prone to error on casual reimplementation.)

Let me give some examples. Maybe it's the whole list, or even a superset. Feel free to say, "I never use that" or "I never use that and I can't see why anybody would". Or say "using a library function for that would be just as verbose as inlining it." Don't feel free to say,"that's trivial". The more trivial something is, the more likely we agree on how it works. I am so tired of writing/pasting these trivial functions, and looking for what they're called in other people's code.

Tables
======

Shallow-copy a table. I'd use a shallow merge-copy too.

If you are interactively working with tables, you have some mechanism to inspect their contents. I don't mean overriding tostring(), I mean "just print what's in the table to the console".  I'll offer for k,v in pairs(t) do print(k,v) end

Given an array, produce a "set". local s={}; for _,v in ipairs(a) do s[v]=true done end return s.

The opposite. Return the keys of a table as an array.

tmap(). tfilter(). The key,value version for tables. Not as widely used as:

imap(), ifilter(). The same thing, except with array-flavor, like ipairs.

The in-place variant of those. No, you can't change the key in tmap_s().[1]

I use a variant of tmap() and tfilter() which accumulates results in an array, usually for table.concat(). I don't know if anybody else does.

I don't use/write these much and hesitate to bring them up: ireduce() and izip(). If you're going to have ireduce, you probably want sum(a,b) and friends.

In verbose-lambda languages, it seems like the world is ending up with list comprehensions etc instead, since map/filter/reduce are bandages over how verbose the in-line imperative looping versions are.

Functions
=========

Given a function f, return a function memoizing f(v). A variant: keep the memoized value in a weak table.

Given a function f and a value v, return function (...) return f(v, ...) end. (Calling it currying makes it seem mysterious; it's called "bind" in luvit, for instance.)

Do it or die: return a function wrapping another function with an assert.

Files
=====

Test whether a file exists. (Mostly important for avoiding overwriting files.)

Read the entire contents of a named file into a string.

Iterate over all the lines of a file which match a pattern. Use captures as the loop variables.

Split a file path into {root, directories, filename}; split a filename.extension.

Objects
=======

In general, please no. A function wrapping the "make an {__index=parent}, set metatable" boilerplate is about as far as I'm willing to go.

I'd like to have "empty proxy with private storage" but this is not so much a basic library thing as "what do you miss from Lua 4.0".

Strings
=======

String interpolation. I despise it, but I find a long-running " ab '"..x.." cd"..y construct worse; you'll notice I forgot to close the quote begun after "ab" which would have been slightly more obvious in " ab'$x cd$y" . I don't care if it's $abc or ${abc} syntax or both. Just stop the quotation/concatenation pain.

Things I do but feel guilty about: xml_escape(), xml_attr_escape(). I don't use the inverse; if I'm consuming XML I for sure need a real parser.

I wish I had some kind of way of shortening the "read a 4k buffer, run some code on all the lines/frames/balanced-expressions, keep the unmatched tail part for the next 4k buffer." That's tedious and error-prone.

Probably a few more. I'm not touching getopt()....

Jay

[1]: BTW, anybody have a good naming convention for mutators, like Scheme's "set!" exclamation point?