Re: paving cowpaths with libraries (was Re: Upstream is not the last word (was Re: [ANN] Lua 5.1.5 (rc1) now available))

[Sean Conner <sean@...>]

It was thus said that the Great Jay Carlson once stated:
> 
> Let me give some examples. Maybe it's the whole list, or even a superset.
> Feel free to say, "I never use that" or "I never use that and I can't see
> why anybody would". Or say "using a library function for that would be
> just as verbose as inlining it." Don't feel free to say,"that's trivial".
> The more trivial something is, the more likely we agree on how it works. I
> am so tired of writing/pasting these trivial functions, and looking for
> what they're called in other people's code.
> 
> Tables
> ======
> 
> Shallow-copy a table. I'd use a shallow merge-copy too.

  I'm iffy on these.  I've found table copies rare, and the two times (in
the same project) I wanted a merge operation, I just wrote my own inline,
since the two tables were quite different.  In fact, here's the code:

  -- --------------------------------------
  -- merge system limits with _DEFAULT
  -- --------------------------------------
  for resource,value in pairs(service.limits) do
    proc.limits.soft[resource] = value
    proc.limits.hard[resource] = value
  end

  for resource,value in pairs(_DEFAULT.limits) do
    if not service.limits[resource] then
      proc.limits.soft[resource] = value
      proc.limits.hard[resource] = value
    end
  end
  
  -- --------------------------------------
  -- now merge the environment
  -- --------------------------------------

  local environment = {}

  for name,value in pairs(_DEFAULT.env) do
    environment[name] = value
  end

  for name,value in pairs(service.env) do
    environment[name] = value
  end

  The first bit, with the resource limits, deals with a proxy table that
wraps calls to setrlimit() within the __newindex method.  Here, we set the
values first, then any not specified specifically get a default value (using
setrlimit() has limits---you can lower, but it's hard to raise).

  The second bit is easier in that the environment table is just a
name/value pairs of strings.  Here, overwriting is fine.  

> If you are interactively working with tables, you have some mechanism to
> inspect their contents. I don't mean overriding tostring(), I mean "just
> print what's in the table to the console".  I'll offer for k,v in pairs(t)
> do print(k,v) end

  I did that at first.  I then wrote my own "pretty print a table" function
that I find easier to read, but it's a bit more complicated than just the
for loop.  

> Given an array, produce a "set". local s={}; for _,v in ipairs(a) do
> s[v]=true done end return s.

  I don't see the reason for this.  The only two values that evaluate to
false are false and nil.  

> The opposite. Return the keys of a table as an array.

  I can see the benefit of this.  I had to do this on a project to report
key/value pairs in a consistent manner (sorted by key).  

> tmap(). tfilter(). The key,value version for tables. Not as widely used as:
> 
> imap(), ifilter(). The same thing, except with array-flavor, like ipairs.
> 
> The in-place variant of those. No, you can't change the key in tmap_s().[1]

  I haven't really done anything that required these functions, but I can
seen a use for some people. 

> I use a variant of tmap() and tfilter() which accumulates results in an
> array, usually for table.concat(). I don't know if anybody else does.

  I don't recall doing anything like this.

> I don't use/write these much and hesitate to bring them up: ireduce() and
> izip(). If you're going to have ireduce, you probably want sum(a,b) and
> friends.

  Again, I don't recall doing anything like this.

> In verbose-lambda languages, it seems like the world is ending up with
> list comprehensions etc instead, since map/filter/reduce are bandages over
> how verbose the in-line imperative looping versions are.
> 
> Functions
> =========
> 
> Given a function f, return a function memoizing f(v). A variant: keep the
> memoized value in a weak table.

  I've done memoizing, but on a table (using __index/__newindex) and not on
a function.

> Given a function f and a value v, return function (...) return f(v, ...)
> end. (Calling it currying makes it seem mysterious; it's called "bind" in
> luvit, for instance.)

  Eh.  While I have used the Y-combinator [1] I haven't wanted a function to
do this.  

> Do it or die: return a function wrapping another function with an assert.

  Eh.  I never did enough Perl programming to want this (or do this).

> Files
> =====
> 
> Test whether a file exists. (Mostly important for avoiding overwriting
> files.)

  Useful.  Even better if it took advantage of any system specific calls (it
can be done using fopen(), but Unix has access()).

> Read the entire contents of a named file into a string.

  I'd rather have a method of indicating to Lua that what I'm giving it is a
"constant" string and thus, it doesn't need to be interred.  I mean, I has
always bugged me that

	lua_pushliteral(L,"constant")

  will cause Lua to copy the string "constant".  It doesn't need to be
copied at all.  I have a few projects that I would benefit from "constant"
strings (I already have the strings in memory, as part of a mmap()'ed file;
there's no need for Lua to copy N megabytes).

  Now, Lua does technically have a way of reading a named file into a
string:

	s = io.open("/tmp/foo","r").read("*a")

but if Lua had a way of specifying constant strings, I could then mmap() a
file into memory, pass the pointer to Lua as a constant string and avoid a
whole mess of IO (and memory allocation, and ... ).

  So, yes (it would be nice) but no (because of the current way Lua works).

> Iterate over all the lines of a file which match a pattern. Use captures
> as the loop variables.

  Hmm, so both f:lines() and s:gmatch() in one ... I could see that as being
somewhat useful in certain contexts.

> Split a file path into {root, directories, filename}; split a
> filename.extension.

  Ah yes, I thought this came up a few times, and it has:

	http://lua-users.org/lists/lua-l/2009-12/msg00641.html
	http://lua-users.org/lists/lua-l/2011-02/msg01592.html
	http://lua-users.org/lists/lua-l/2011-11/msg00125.html

  Now, given the above, what's a use case senario for this?

> Objects
> =======
> 
> In general, please no. A function wrapping the "make an {__index=parent},
> set metatable" boilerplate is about as far as I'm willing to go.
> 
> I'd like to have "empty proxy with private storage" but this is not so
> much a basic library thing as "what do you miss from Lua 4.0".

  I don't really have an opinion on this, so my default position is "keep
the status quo".

> Strings
> =======
> 
> String interpolation. I despise it, but I find a long-running " ab
> '"..x.." cd"..y construct worse; you'll notice I forgot to close the quote
> begun after "ab" which would have been slightly more obvious in " ab'$x
> cd$y" . I don't care if it's $abc or ${abc} syntax or both. Just stop the
> quotation/concatenation pain.

  I would vote this down---how do you escape the string interpolation
characters?  Do strings in '' not undergo interpolation?  Or was it ""?  Do
strings in [[ ]] undergo interpolation?  All strings?  Sometimes?  

  Sorry, if I want Perl or bash, I know where to find it.

> Things I do but feel guilty about: xml_escape(), xml_attr_escape(). I
> don't use the inverse; if I'm consuming XML I for sure need a real parser.
> 
> I wish I had some kind of way of shortening the "read a 4k buffer, run
> some code on all the lines/frames/balanced-expressions, keep the unmatched
> tail part for the next 4k buffer." That's tedious and error-prone.

  I'm not sure I understand this.  Lua (and C) provide different levels of
buffering (by character, by line, by size).

> Probably a few more. I'm not touching getopt()....

  I'm still not satisfied with any of the getopt() implementations,
including my own ... 

> [1]: BTW, anybody have a good naming convention for mutators, like
> Scheme's "set!" exclamation point?

  Nope.  

  -spc (I'm recently finding myself hating abstration just for abstraction's
	sake)

[1]	A way to write recursive anonymous functions