Re: Simple configurable expression parser (in pure Lua)

[Lorenzo Donati <lorenzodonatibz@...>]

On 20/06/2022 08:03, Thijs Schreijer wrote:
> On 14 Jun 2022, at 13:00, Luiz Henrique de Figueiredo
> <lhf@tecgraf.puc-rio.br<mailto:lhf@tecgraf.puc-rio.br>> wrote:
>
> can it parse operators that are not Lua operators, i.e. can I define,
> for example, a "&=" or "===" binary operator?
>
> No, my code can only handle Lua syntax.
>
> Not sure it is helpful, but Penlight has with a lexer;
> https://lunarmodules.github.io/Penlight/libraries/pl.lexer.html
Thanks for this pointer, Thijs.

Unfortunately it's not immediately useful in my use case (I need more of 
a generic configurable parser, than a simple lexer).

However maybe I could hack it to make it work for my purposes (depending 
on how contrived its code is).

The major obstacle for me it's the very limited time I have got to 
dedicate to the effort.

My ad-hoc solutions work quite well. My problem is that I'm getting used 
to this evil workflow:

1. Ok, I need to parse an expression (Lua/C/C++/Latex/etc.) and maybe
rewrite it in an "equivalent" (for some definition of equivalent) or 
modified form in another language (Lua/C/C++/Latex). Or maybe I have to 
generate a random expression (Lua/C/C++/Latex) of given 
structure/complexity.

2. Mmmh, I think I did that before. Let's look at my older code. Ah! 
Here it is something similar!

3. Copy-Modify-Paste and Test until it works.

All this because I've never found the time to consolidate all that code 
(and probably I never will).

This is getting ludicrous and my "professional developer" brain hurt any 
time I'm forced to do that. If it were for a couple of throwaway script, 
meh!, who would care. The issue is this turns out to be a often 
recurring problem.

Just to be more specific: I usually write the tests (or other documents) 
for my students as LaTeX fragments (with embedded Lua code) that a set 
of tools (written by me in Lua) put together to obtain the final PDF for 
the test.

These tools have lots of useful functionalities, such as randomizing the 
position of questions and answers and, upon further configuration, also 
randomization of part of the content.

This allows me to produce a number of different versions of the same 
test, but still maintaining its general content and difficulty level.

This is quite more efficient than using a simple word processor (even in 
conjunction with embedded scripting), since through Lua+Latex I have the 
power to render the document exactly as I want programmatically, also 
handling complex mathematical notations or electronic circuit schematics.

However, this has drawbacks: with every version I have a different 
solution set, and during the correction phase it's just too easy to mark 
the right answer as good or vice-versa. To save time I correct them 
putting my brain in "dumb-but-quick" mode: I simply check them against a 
"cue sheet" (specific for each version) with the right answers that my 
tools automatically generate.

However that needs a way to enter some formulas in one language Syntax 
(e.g. C), converting them to Lua for evaluation and then to Latex for 
rendering. All this to avoid more failure points I would have if I had 
to write the formulas in three different syntaxes manually.

Hope this helps framing the issue.


> Regards Thijs

Cheers!

-- Lorenzo.