Re: Re[2]: Still cryptic OOP syntax

[David Olofson <david@...>]

On Sunday 30 January 2005 14.19, skaller wrote:
[...]
> It would actually be cleaner to get rid of operators
> from the core completely. Functions are good enough.
> Just call the 'add' function listed in the table
> for operator '+'. Or operator '*' ... or any operator.

Well... I just reworked the operator subsystem of EEL (a language 
rather similar to Lua in many ways; http://eel.olofson.net ) - and I 
realized it's not quite that simple. :-)

In theory, you could just construct a multidimensional array of 
operator calls; [all_types][all_operations][all_types]. I did 
something similar for the simple types (integer, reals, booleans; 
anything that doesn't carry an object reference) in the form of a 
large single level switch that decodes values combined from the 
operator and the operand types. Fast and simple, and yet makes it 
trivial to implement things like "Pascal style divison" (integer / 
integer ==> real, rather than integer).

The problems gather when you get to operations on objects...

First, what about casting and/or operations on mixed types? The simple 
approach is to always cast the right operand to the type of the left 
operand, before even looking at the operator. It works (AFAIK, this 
is how quite a few languages do it), but it can become very expensive 
in some cases. For example, consider this EEL code:

 v = vector [1, 2, 4, 2, 1];
 v = v * .5;

This will create a vector (compact and fast "raw" array of static 
typed values) of 5 values and then multiply the vector by .5. As it 
is, this calls the MUL metamethod (EEL metamethods are most similar 
to Lua tag methods, I think...) - ie a member of the vector class - 
with the vector itself (an objref value) and the real value .5 (a 
real value) as operands. The operation is simple and fast, and could 
even be optimized with SIMD instructions if desired.

(Actually, with that syntax, the VM creates an empty clone of v to 
receive the result, rather than actually having the metamethod modify 
the original instance. That's not really relevant to this discussion, 
though.)

With the "always cast to type of left hand operand" method, this 
results in the value .5 being cast to a vector (single item - or 
filled with .5, but that would require an extra argument to the 
constructor), which means some temporary object overhead. What's 
perhaps more interesting is that you've just forced every constructor 
to also serve as a cast operator, or alternatively, you get even more 
overhead as a result of creating empty objects and then filling them 
in with another method(s), including or via some casting mechanism.

The way it is now in EEL, casting is left to the operator metamethods 
of objects, but so far, I think that's the best place to do it. You 
can do it the simple and "slow" way, just calling some toolkit 
function (like eel_v2l() or eel_v2d()) to cast to/extract what you 
need, or you can do explicitly handle one or more types specially. 
For example, the EEL vector class handles other vectors specially to 
allow optimized vector/vector operations. That mechanism can be used 
for string concatenation, adding files to archives or whatever - and 
it's available both to the VM and any classes registered by host 
applications or extensions at run time.


The second major problem with operations on objects is when doing 
things like

 v = 1 / v;

Now what? You can't use the DIV method of v, because that can only do 
self / value; note value / self. And you can't just cast v to a value 
and send it off to the DIV operator for simple types, because the 
expected result is a vector with all values of v inverted; not the 
inverse of whatever you get if you cast a vector to a single value. 
(Which is not valid in EEL currently, as I don't see any single 
obvious operation to map to that interface.)

One solution is to implement both DIV and "INVDIV" methods in all 
classes that you want to support the / operator. (Well, you don't 
*have* to implement both if only one makes sense...) The interface 
could be in the form of a different metamethod argument layout or of 
dual metamethods for non-commutative operators. The net result is 
essentially the same.


Anyway, whatever you do, you're basically just pushing the problem 
around, rather than avoiding it. :-) AFAIK, there's no way around it, 
without seriously crippling the language in terms of data type 
interperability.


Now, obviously, if the language is set in stone, you must implement it 
correctly, so your options become significantly fewer. I still have 
the luxury of playing around with a brand new language that I can 
change to suit my neads without pissing off a large community of 
users. :-)


> > As for "the whole thing": Dylan shows quite well that it can be 
done
> > even for an algol-like syntax. I would not want to even come close 
to
> > that kind of complexity for lua, though.
> 
> Contrarily, generalising Lua to allow such stuff should
> probably *simplify* the core, for example as above.

Did that in EEL (the VM doesn't mess with casting when doing 
operations on objects; that's passed on to the class implementations) 
- but that means you just move the complexity from one place (in the 
core) into every single class - built-in or user defined - that 
implements any operators.

In the case of EEL, that's actually a feature, rather than a problem, 
but anyway; one has to realize the complexity doesn't magically go 
away, no matter what.

However, the "leave it to the implementations" approach does have an 
advantage in terms of complexity: When you hit type/operator/type 
combinations that are invalid, the VM quickly (as usual) calls the 
operator implementation of one of some object, which is even quicker 
to conclude that "I can't do this" and throw an exception. Simple and 
effective.


> BTW: Felix does this. The parser translates
> 
>  a + b
> 
> into
> 
>  apply(add, (a,b))
> 
> The Felix language core doesn't know what an operator is.
> It knows how to apply a function though, and that is enough.

Kewl.


> In Felix this doesn't impact run time performance,
> in Lua eliminating +-*/ opcodes and using a function
> call instead might.. but it would be much easier to
> extend .. YMMV here.

Not sure about the performance aspect...

It might impact performance to some extent in EEL, and I believe that 
applies to Lua (which seems to implement operators in a very similar 
manner), but I suspect that's mostly because the normal "calling 
machinery" handles procedures (no result), functions (result), 
required arguments, optional arguments and tuple arguments.

Looking at EEL's exception handling, I realize that the complexity and 
cost is really in dealing with results and arguments; not the actual 
function calling. In EEL, it's done the simplest way I could think 
of; by compiling the 'try' and 'except' blocks as lightweight 
functions, called by a special VM instruction that installs the 
'except' block as a "catcher" before calling the 'try' block.

The interesting part is that the exception block functions have 
hardwired call prototypes, so I could call them using a substantially 
simplified version of the normal function call code. (As it is, I 
just use the same inline C function for all calling in the VM.)

Special-casing frequently used call prototypes (like "function with 
two required arguments") would serve both as a general optimization 
and a shortcut for "operator" functions. Might work for the Lua VM as 
well.


//David Olofson - Programmer, Composer, Open Source Advocate

.- Audiality -----------------------------------------------.
|  Free/Open Source audio engine for games and multimedia.  |
| MIDI, modular synthesis, real time effects, scripting,... |
`-----------------------------------> http://audiality.org -'
   --- http://olofson.net --- http://www.reologica.se ---