Re: Questions about writing a C library

[Phil Bewig <pbewig@...>]

Perhaps more discussion about me and my project is in order.

Me:  I cut my teeth on MSBASIC, where variable names could
be a single letter or a single letter followed by a single
digit; the interpreter was read into the 2K memory from a
cassette tape.  My first pocket calculator cost $225 back
when Motel 6 was called that because rooms were $6 per
night, nationwide; I could have bought a calculator for
$150, but it didn't have a square root key.  I was the
local wizard at my college computing center on setting up
the punch card templates that controlled the tab key on the
card punch machine.  I just counted books about seventeen
different languages in my personal library, from fortran to
ocaml, and that doesn't include any on PL/SQL (those are at
work).  I refuse to use perl, and use c only when necessary,
though I am competent with both.  My favorite general-purpose
language is probably scheme, assuming I can find libraries
for my current task, though I've probably spent more of my
life programming awk than any other language, and my most-
used language in the last six months is visual basic.  I run
a dual-boot Win98/debian box at home and use W2K and Solaris
at work.  I prefer vi to emacs, but I still use ed to write
edit scripts, as I just did last week.

I am one of a half-dozen programmers working on a large
database application using the Oracle database and languages
SQL, ProC, and PL/SQL.  PL/SQL is a scripting-type language,
interpreted via a byte-code compiler, with such features as
garbage collection and associative arrays, and of course it
offers SQL cursors to access the database.  We use PL/SQL a
lot.  And we hate it, for a variety of reasons that aren't
pertinent here.  I've been on a mission for several months
to find a language that can replace PL/SQL, and am currently
considering whether lua will work.

My first impressions of lua are good.  It seems to have a
strongly functional flavor, which I like.  There is a luaSQL
version that provides native support for Oracle, which I
need.  And the easy interface to c is important because of
the ProC code that is part of the application.  At this
point I see no negatives of lua.  My next steps are to
learn more about lua and then convince my fellow programmers
to make the change, which I realize will be the hard part
(they agree they want to change -- the really hard part
will be getting agreement on what to change to -- though
they did all agree they didn't like my suggestion to use
scheme -- strongly).  I asked earlier about static variables
because that is a specific item that we have talked about.

I chose linked lists as my vehicle to learn lua because I
have found them useful in a variety of languages in the
past.  I'm not as concerned about efficiency as my earlier
posting may have sounded, although I'm never opposed to
efficiency, either.  This discussion reminds me of the
pitched battles surrounding the addition of arrays to the
haskell programming language.  The computer scientists
wanted the language to remain purely functional, and saw
arrays as impossibly imperative.  The programmers took the
two-orders-of-magnitude improvement in runtimes as a
sufficient indication of the need for arrays.  Haskell, as
a practical programming language rather than a theoretical
one, still hasn't recovered.  To answer the question that
a couple of people haven't quite asked, my investigation of
lists is probably about three-quarters just learning how
the language works and one-quarter about whether c-native
lists are useful in lua.

I suppose it's fair to say that the three fundamental data
structures in computing are arrays, lists, and hash tables
(I still remember how amazed I was when I first learned
about hash tables, almost thirty years ago -- they seemed
like magic them, and sometimes still do today).  It's also
probably fair to say that most imperative languages are
built around arrays, most functional languages are built
around lists, and most scripting languages are built around
hash tables.  And even further, it's probably fair to say
that all three data structures can do a reasonable job of
simulating each other, so whichever one your language gives
you, it's always possible to get your task written.  But
the three are fundamentally different, with varying strengths
and weaknesses, and there is no reason to say that lua uses
hash tables, so linked lists are somehow unclean.

I note that all three people who responded on the mailing
list objected to my use of the list data structure.  That
surprises me.  Lists are intrinsically ordered, tables are
intrinsically unordered.  Lists take one object per node,
tables take two objects, key and value, per node.  Lists
and tables are just ... different.  Is there some reason
lua people don't like lists?  Is there some unmentionable
tar pit you are all trying to steer me away from?

RLake suggests I learn more about lua before I start trying
to mold it to fit my ideas of what a language should be.
That's a fair point.  On the other hand, I'm highly
experienced with awk, another language based on hash tables,
which I've used to simulate such diverse data structures as
deques, directed acyclic graphs, and balanced binary trees
(don't ask -- I'll never do it again, I swear).  And I'm
also highly experienced with scheme, and I recognize the
strongly functional flavor of lua, so I think lists would
fit nicely, expecially if there is an efficient
implementation based on c-native lists.  I hope my
knowledge of computing is sufficient to help me use lua
wisely.  At this point in my investigation of lua, I'm
mostly looking for anything that might disqualify it for
my project, so I don't waste time if lua's not the right
solution for me, and I can go search for something else.

As to the question of freeing cons cells:  Assume variable
lst points to a list of ten thousand nodes.  Simply
assigning list.null to lst should free all those nodes and
everything they point to, assuming nobody else is holding
a copy.  And that raises a question:  how does lua decide
when an object is free and subject to garbage collection?

And no, RLake, I don't take your comments personally.
Your comments are obviously sincere and well-meant, and
I appreciate your interest in my success.

A note to Briah Hook:  No, I'm not particularly concerned
about low-level efficiency, except in abstract terms.  Any
use of lua in my project will doubtless be bound by the
runtime of the cursor that interacts with the database, to
the point where almost any language will be efficient
enough.  And I have never written a linked list library in
c; it's easy to just hard-code lists where I need them.

Another respondent has suggested via private email that I
look at the lauxlib.h and lauxlib.c files for samples of
how to do what I want, and has answered some specific
questions.  One that he didn't answer is how to make
type(obj) respond "list" if obj is a list.  Can anyone
else answer that question?  That question has caused some
problems with scheme's generative types, now answered by
srfi-9.

I have mentioned a few times that I feel a strongly
functional flavor in lua.  I suspect that in addition
to lists, lua would be improved with some library support
for higher-order functions.  Function composition,
currying, identity and negation functionals might all
be useful.  In a parallel universe far from here, I am
currently finalizing srfi-40 for streams; would an
explicit delay/force, as in scheme, fit the language?  
Closures mean that lua has the tools to be programmed in
continuation-passing style; has anybody done that and
survived?  Please don't think I am trying to force lua
to fit my idea of the perfect language.  I'm not.  I'm
just responding to what I see when I look at the language.
Hopefully my comments as an outsider might trigger those
of you who are experts to add features and expressive
power to the language, wisely, in keeping with its own
unique character.

This has been a fascinating and interesting discussion,
hopefully not to end here.  Thanks to all.

Phil