[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: Lua 5.2
- From: Jim Pryor <lists+lua@...>
- Date: Thu, 26 Nov 2009 11:16:32 -0500
On Thu, Nov 26, 2009 at 05:01:57PM +0100, Richter, Jörg wrote:
> I'm currently in need to yield from iterator functions. From
> http://lua-users.org/lists/lua-l/2009-08/msg00540.html it seems
> like Lua 5.2 can do it out of the box.
>
> Before bothering with Coro or RVM I'd like to know if the Lua
> authors have an estimated release date for Lua 5.2? Or alternatively
> a statement that it will take at least some more months.
There's a patch that's simpler than Coco or RVM that just enables yielding from iterator
functions.
It's described here: http://lua-users.org/wiki/YieldableForLoops
also here: http://lua-users.org/wiki/LuaPowerPatches
However, I wasn't able to find the patch (or any webserver) at the
designated site. I don't know where Rici Lake is now hosting this, if
anywhere.
However, I did find a version of the patch through Google archive or the
Wayback Machine. It's below. I may have tweaked it to apply against Lua
5.1.4; at any rate, this version does work against Lua 5.1.4 and it's
giving me no problems.
I trust Rici has no objection to our keeping this patch circulating.
Yieldable For Loop
* Backwards compatible: yes if LUA_COMPAT_TFORLOOP defined
* Lines changed/added/deleted:
* Author: RiciLake
* Last update: 2007-Sep-01
* [Download for 5.1.2]
Modifies the code generator so that the iterator in for ... in loops can call yield. Details and test code are available at YieldableForLoops.
Note that the current version of the patch orders op codes so as to maintain binary compatibility with compiled Lua scripts, if LUA_COMPAT_TFORLOOP is defined in
luaconf.h. This adds a few instructions to the VM, and quite a bit of complexity to the patch, which would otherwise only be about 25 lines modified.
Yieldable For Loops
Changing Lua so that "anything can yield" is (probably) desirable, but
it's a long term project. In the meantime, I find it irritating that the
iterator function in a for loop is not allowed to yield; it makes it messy
to write simple responder loops where the iterator might be, for example,
an asychronous input function.
Instead of just being able to write:
for msg in eachmsg() do
-- handle msg
end
-- end of messages, clean up
you need:
repeat
local msg = getmsg()
if msg == nil then break end
-- handle msg
until false
-- end of messages, clean up
However, it is very simple to get the first code sample to work. It is
only necessary to split the TFORLOOP VM operation into two opcodes. The
first one sets up an ordinary Lua call and then falls into the OP_CALL
implementation. The following op code does a conditional move and branch
based on the first value returned by the first op code.
Some very rough testing seems to show that performance is actually
slightly improved by this change, although the results are not definitive.
I suppose that is because the VM can handle the call without recursing,
making up for the overhead of an extra opcode.
At any rate, the patch is at [1].
diff -urN lua-5.1.2/src/ldebug.c lua-5.1.2-yieldpatch/src/ldebug.c
--- lua-5.1.2/src/ldebug.c Fri Mar 23 17:06:19 2007
+++ lua-5.1.2-yieldpatch/src/ldebug.c Thu Jul 12 19:38:32 2007
@@ -390,16 +390,21 @@
check(b < c); /* at least two operands */
break;
}
+#ifdef LUA_COMPAT_TFORLOOP
case OP_TFORLOOP: {
check(c >= 1); /* at least one result (control variable) */
checkreg(pt, a+2+c); /* space for results */
if (reg >= a+2) last = pc; /* affect all regs above its base */
break;
}
+#endif
case OP_FORLOOP:
case OP_FORPREP:
checkreg(pt, a+3);
/* go through */
+ case OP_TESTNIL:
+ checkreg(pt, a+1);
+ /* go through */
case OP_JMP: {
int dest = pc+1+b;
/* not full check and jump is forward and do not skip `lastpc'? */
@@ -407,6 +412,11 @@
pc += b; /* do the jump */
break;
}
+ case OP_TFORCALL:
+ check(a >= b);
+ check(b == 3);
+ check(c >= 1);
+ /* go through */
case OP_CALL:
case OP_TAILCALL: {
if (b != 0) {
@@ -531,9 +541,14 @@
return NULL; /* calling function is not Lua (or is unknown) */
ci--; /* calling function */
i = ci_func(ci)->l.p->code[currentpc(L, ci)];
- if (GET_OPCODE(i) == OP_CALL || GET_OPCODE(i) == OP_TAILCALL ||
- GET_OPCODE(i) == OP_TFORLOOP)
+ if (GET_OPCODE(i) == OP_CALL || GET_OPCODE(i) == OP_TAILCALL
+#ifdef LUA_COMPAT_TFORLOOP
+ || GET_OPCODE(i) == OP_TFORLOOP
+#endif
+ )
return getobjname(L, ci, GETARG_A(i), name);
+ else if (GET_OPCODE(i) == OP_TFORCALL)
+ return getobjname(L, ci, GETARG_A(i - 3), name);
else
return NULL; /* no useful name can be found */
}
diff -urN lua-5.1.2/src/lopcodes.c lua-5.1.2-yieldpatch/src/lopcodes.c
--- lua-5.1.2/src/lopcodes.c Tue Nov 8 19:45:36 2005
+++ lua-5.1.2-yieldpatch/src/lopcodes.c Thu Jul 12 19:38:32 2007
@@ -47,11 +47,15 @@
"RETURN",
"FORLOOP",
"FORPREP",
+#ifdef LUA_COMPAT_TFORLOOP
"TFORLOOP",
+#endif
"SETLIST",
"CLOSE",
"CLOSURE",
"VARARG",
+ "TESTNIL",
+ "TFORCALL",
NULL
};
@@ -93,10 +97,14 @@
,opmode(0, 0, OpArgU, OpArgN, iABC) /* OP_RETURN */
,opmode(0, 1, OpArgR, OpArgN, iAsBx) /* OP_FORLOOP */
,opmode(0, 1, OpArgR, OpArgN, iAsBx) /* OP_FORPREP */
+#ifdef LUA_COMPAT_TFORLOOP
,opmode(1, 0, OpArgN, OpArgU, iABC) /* OP_TFORLOOP */
+#endif
,opmode(0, 0, OpArgU, OpArgU, iABC) /* OP_SETLIST */
,opmode(0, 0, OpArgN, OpArgN, iABC) /* OP_CLOSE */
,opmode(0, 1, OpArgU, OpArgN, iABx) /* OP_CLOSURE */
,opmode(0, 1, OpArgU, OpArgN, iABC) /* OP_VARARG */
+ ,opmode(0, 1, OpArgR, OpArgN, iAsBx) /* OP_TESTNIL */
+ ,opmode(0, 1, OpArgU, OpArgU, iABC) /* OP_TFORCALL */
};
diff -urN lua-5.1.2/src/lopcodes.h lua-5.1.2-yieldpatch/src/lopcodes.h
--- lua-5.1.2/src/lopcodes.h Tue Mar 14 19:04:44 2006
+++ lua-5.1.2-yieldpatch/src/lopcodes.h Thu Jul 12 19:38:32 2007
@@ -197,18 +197,23 @@
if R(A) <?= R(A+1) then { pc+=sBx; R(A+3)=R(A) }*/
OP_FORPREP,/* A sBx R(A)-=R(A+2); pc+=sBx */
+#ifdef LUA_COMPAT_TFORLOOP
OP_TFORLOOP,/* A C R(A+3), ... ,R(A+2+C) := R(A)(R(A+1), R(A+2));
if R(A+3) ~= nil then R(A+2)=R(A+3) else pc++ */
+#endif
OP_SETLIST,/* A B C R(A)[(C-1)*FPF+i] := R(A+i), 1 <= i <= B */
OP_CLOSE,/* A close all variables in the stack up to (>=) R(A)*/
OP_CLOSURE,/* A Bx R(A) := closure(KPROTO[Bx], R(A), ... ,R(A+n)) */
-OP_VARARG/* A B R(A), R(A+1), ..., R(A+B-1) = vararg */
+OP_VARARG,/* A B R(A), R(A+1), ..., R(A+B-1) = vararg */
+OP_TESTNIL,/* A sBx if R(A+1) ~= nil then R(A)=R(A+1); pc+=sBx */
+OP_TFORCALL/* A B C R(A), ..., R(A+C-2) := R(A-3)(R(A-2), R(A-1)); */
+
} OpCode;
-#define NUM_OPCODES (cast(int, OP_VARARG) + 1)
+#define NUM_OPCODES (cast(int, OP_TFORCALL) + 1)
@@ -217,6 +222,8 @@
(*) In OP_CALL, if (B == 0) then B = top. C is the number of returns - 1,
and can be 0: OP_CALL then sets `top' to last_result+1, so
next open instruction (OP_CALL, OP_RETURN, OP_SETLIST) may use `top'.
+
+ (*) In OP_TFORCALL, B must be 3 and C may not be 0.
(*) In OP_VARARG, if (B == 0) then use actual number of varargs and
set top (like in OP_CALL with C == 0).
diff -urN lua-5.1.2/src/lparser.c lua-5.1.2-yieldpatch/src/lparser.c
--- lua-5.1.2/src/lparser.c Fri Mar 23 17:06:30 2007
+++ lua-5.1.2-yieldpatch/src/lparser.c Thu Jul 12 20:32:14 2007
@@ -1045,7 +1045,7 @@
/* forbody -> DO block */
BlockCnt bl;
FuncState *fs = ls->fs;
- int prep, endfor;
+ int prep;
adjustlocalvars(ls, 3); /* control variables */
checknext(ls, TK_DO);
prep = isnum ? luaK_codeAsBx(fs, OP_FORPREP, base, NO_JUMP) : luaK_jump(fs);
@@ -1055,10 +1055,16 @@
block(ls);
leaveblock(fs); /* end of scope for declared variables */
luaK_patchtohere(fs, prep);
- endfor = (isnum) ? luaK_codeAsBx(fs, OP_FORLOOP, base, NO_JUMP) :
- luaK_codeABC(fs, OP_TFORLOOP, base, 0, nvars);
- luaK_fixline(fs, line); /* pretend that `OP_FOR' starts the loop */
- luaK_patchlist(fs, (isnum ? endfor : luaK_jump(fs)), prep + 1);
+ if (isnum) {
+ luaK_patchlist(fs, luaK_codeAsBx(fs, OP_FORLOOP, base, NO_JUMP), prep + 1);
+ luaK_fixline(fs, line); /* pretend that `OP_FOR' starts the loop */
+ }
+ else {
+ luaK_codeABC(fs, OP_TFORCALL, base + 3, 3, nvars + 1);
+ luaK_fixline(fs, line); /* pretend that `OP_FOR' starts the loop */
+ luaK_patchlist(fs, luaK_codeAsBx(fs, OP_TESTNIL, base + 2, NO_JUMP), prep + 1);
+ luaK_fixline(fs, line); /* pretend that `OP_FOR' starts the loop */
+ }
}
diff -urN lua-5.1.2/src/luaconf.h lua-5.1.2-yieldpatch/src/luaconf.h
--- lua-5.1.2/src/luaconf.h Sat Mar 24 03:01:55 2007
+++ lua-5.1.2-yieldpatch/src/luaconf.h Thu Jul 12 19:38:32 2007
@@ -312,7 +312,12 @@
*/
#define LUAI_GCMUL 200 /* GC runs 'twice the speed' of memory allocation */
-
+/*
+@@ LUA_COMPAT_TFORLOOP controls VM compatibility with 5.1 for..in loops
+** CHANGE it (undefine it) if you don't care about being able to run
+** precompiled 5.1 scripts
+*/
+#define LUA_COMPAT_TFORLOOP
/*
@@ LUA_COMPAT_GETN controls compatibility with old getn behavior.
diff -urN lua-5.1.2/src/lvm.c lua-5.1.2-yieldpatch/src/lvm.c
--- lua-5.1.2/src/lvm.c Mon Mar 26 14:32:38 2007
+++ lua-5.1.2-yieldpatch/src/lvm.c Thu Jul 12 20:32:14 2007
@@ -581,6 +581,19 @@
pc++;
continue;
}
+ case OP_TESTNIL: {
+ if (!ttisnil(ra+1)) {
+ setobjs2s(L, ra, ra+1);
+ dojump(L, pc, GETARG_sBx(i));
+ }
+ continue;
+ }
+ case OP_TFORCALL: {
+ setobjs2s(L, ra+2, ra-1);
+ setobjs2s(L, ra+1, ra-2);
+ setobjs2s(L, ra, ra-3);
+ /* FALLS THROUGH */
+ }
case OP_CALL: {
int b = GETARG_B(i);
int nresults = GETARG_C(i) - 1;
@@ -676,6 +689,7 @@
dojump(L, pc, GETARG_sBx(i));
continue;
}
+#ifdef LUA_COMPAT_TFORLOOP
case OP_TFORLOOP: {
StkId cb = ra + 3; /* call base */
setobjs2s(L, cb+2, ra+2);
@@ -692,6 +706,7 @@
pc++;
continue;
}
+#endif
case OP_SETLIST: {
int n = GETARG_B(i);
int c = GETARG_C(i);
--
Profjim
profjim@jimpryor.net
