Re: [ANN] Penlight Libraries, First release

[steve donovan <steve.j.donovan@...>]

On Wed, May 13, 2009 at 8:24 AM, David Manura <dm.lua@math2.org> wrote:

> List:map, List:mapm, List:partition, and List:slice return new lists.
> I think at least the map functions should not.

As I understand, you mean that tablex.map does not return a List.  The
idea is not to make the List interface obligatory, or an irritating
side effect.  I did think that tablex.map could ensure  that the
result has the same metatable as its input, but I'm not sure about it
now.

> pl.list is not to be confused with the pl.array module, which
> implements two-dimensional arrays

Which does suggest that 'array' is not the correct word to use.
Obviously they are not matrices, so we need a term for 'rank 2 arrays'
which does not sow confusion.  Perhaps 'array2d' would be better.  It
is really just a module that provides higher rank versions of map and
reduce.

On that subject, David and I have been talking about a more compact
and suggestive notation for generalized map-reduce operation. He
suggested I look into APL/J, which is completely array-oriented.
However, these are eccentric languages, in particular evaluation is
strictly right-to-left, which perhaps seems natural to a functional
person. But it did suggest the power of compact notation.

 The idea is to compile operator strings efficiently into these
operations.  If A means 'apply operator string', then it could look
liike this. The left and right ranks are explicit, . means rank 1, :
means rank 2, otherwise rank 0.

A('.*. [+]',u,v)  -- dot product of two rank-1 tables   (dot-star-dot)

A('.*',u,a)    --- same as { x*a | x in u }   (dot-star)

A('*.',a,u)    -- same as {a*x | x in u }   (star-dot)

And then '.*:' (dot-star-colon) would mean element-wise multiplication
of the rows of the 2d array with the 1d array.

As for generalized reduction, [+,] would mean 'reduce columns with +',
 [,*] 'reduce rows with *', and [+,*] would be a full reduce with two
operators.

As with any notation, the aim is to get an efficient, compact and
consistent way to represent operations, and would perhaps be more
memorable than the current zoo of
map,map2,reduce,reduce_rows,reduce_cols, etc.  Also this can produce
efficient code with dynamic compilation techniques, especially if
there are map-reduce chains.

I know this isn't my usual style, which is to start with a reference
implementation, but I'm curious about what people think, and most
importantly, whether it is promising enough to pursue further.

steve d.