Re: [PROPOSAL 5.4] alternate way of format/pack/unpack (in the string library)

["Pierre Chapuis" <catwell@...>]

May 26, 2017 11:55 PM, "Sean Conner" <sean@conman.org> wrote:

> I don't find the pack/unpack minilanguage all that bad, per se. Lowercase
> letters are signed quantities, uppercase unsigned and there's some mneumonic
> meaning to the letters used. But it can get silly (sample from an SMPP
> parser):
> 
> result.service_type,
> result.source.addr_ton,
> result.source.addr_npi,
> result.source.addr,
> result.dest.addr_ton,
> result.dest.addr_npi,
> result.dest.addr,
> result.esm_class,
> result.protocol_id,
> result.prority,
> result.schedule_time,
> result.validity_period,
> result.registered_delivery,
> result.replace_if_present,
> result.data_coding,
> result.sm_default_msg_id,
> result.message =
> string.unpack(">z I1 I1 z I1 I1 z I1 I1 I1 z z I1 I1 I1 I1 s1",blob,13)
> 
> It was hard to debug, and the obvious solution:
> 
> result.service_type,pos = string.unpack(">z",blob,pos)
> result.source.addr_ton,pos = string.unpack(">I1",blob,pos)
> result.source.addr_npi,pos = string.unpack(">I1",blob,pos)
> --- and so on
> 
> just *feels* a lot slower to me. One could try to create another
> minilanguage for this, and I've tried, but I haven't created one that I
> like (for me, the *same* language should create both encoder and decoder).

I really liked the idea behind Ignacio Burgueño's solution for that issue in his ZX Spectrum emulator [1].

In general, I feel like this is the same issue as for text parsing: there is a simple minilanguage for simple things (patterns for text and string.pack / string.unpack for binary), which is also found in other languages (for instance in re and struct in Python). For more complicated things an approach where Lua objects and/or formal grammars can be used makes sense. For text we have LPeg, we could have something for binary. The simple minilanguages belong in core Lua, and as of now LPeg does not (even though I would personally like to see it become part of an extended standard library, but that's another issue).

I don't feel the need for something like this for encoding though. Maybe that's just me but I have always found that outputing a specified format is easier than parsing it and does not necessarily deserve as much tooling.

[1] https://github.com/ignacio/luagleck/blob/bd342a920bc67de48764327c9e881d3e15bc3a4d/file_format/sna.lua#L25-L67

-- 
Pierre Chapuis