Re: string.pack with bit resolution

Also I would be pleased to see Lua listed in at least one category of:

https://en.wikipedia.org/wiki/List_of_programming_languages_by_type

And have the data model extended to support efficient computing for arrays, matrixes, or N-dimensional data (possibly with sparse data).

The data sources could be from various data models (including RDF triples): database engines working with triples are now also widespread and booming (sometimes replacing RDBMS and SQL). Could it be possible to include Lua scriptlets in their query languages? For now there's active work using languages like R, Ada, _javascript_ and allowing them to process non-relational data. As well efficient data representation for geography and 3D modelling/simulation and in physics/engineering (fluid mechanics, meteorology...).

Lua is still not efficient enough with only its basic discrete types and "flat" tree-like data structures. OpenCL is a good model we could try to mimic to allow closer integration. Processing sparse data (e.g. from RDF triples or geographic systems) with a convenient and compact data representation may be a research goal.

Next will come the processing of data with estimations or probabilistic data, and fuzzy logic: a single number is not enough, they also need vector formats to be efficiently processed. And a new set of arithmetic or logical operations with their own semantics, still not covered by IEEE 754.

Le dim. 20 oct. 2019 à 16:56, Philippe Verdy <verdy_p@wanadoo.fr> a écrit :

Just look at this interesting list for floatting point formats (intended for data interchange):
https://en.wikipedia.org/wiki/IEEE_754
This is something you should keep in mind
Also consider the additional formats used in FPUs for x86 and ARM (in discrete instructions)

And vector formats in SSE/AVX and APU/GPU (for OpenCL) as they are now widespread
https://en.wikipedia.org/wiki/OpenCL#Conformant_products

It would be interesting to have Lua ported to work within an OpenCL engine and then be able to use GPUs and other vector extensions (which are very userful for massive data processing, which for now are still processed in legacy languages like FORTRAN, or proprietary mathematical languages and libraries).

Such work has been done for porting apps written in C, C++, Java, Perl, Python, .Net/CLI... but still not Lua and allow their parallelization. Note that I do not propose here a native parallelization in Lua, something that would be another proposal. ANSI C is just a very limited goal for Lua and it does not ease its safe integration in existing application servers or SQL engines (which could host Lua scriptlets, instead of using only Java or some old specific GL language like PL/SQL), except in specific containers running on separate isolated servers (but with poor/slow interaction, as this also requires them to communicate over some networking API, including REST API with response delays, and this is not acceptable for embedded devices and cause complex security concerns)

Le dim. 20 oct. 2019 à 16:21, bil til <flyer31@googlemail.com> a écrit :
Hi Phillipe,
thank you for your answer.

So you mean IEEE float numbers, you want to have even more there? So which
ones exactly?

Just please understand that it does not make too much sense to support IEEE
numbers, which are NOT supported by the lua_number type ... and I think it
is wise of lua to concentrate thus on the "ANSI C" types. Just for
reasonable future compatibilty, I would recommend to add "long double /
float128", as this is included already in ANSI C (though of course not yet
available on LUA_32BITS / LUA_64BITS), and a special wish for greedy "small
string optimizers" like for "short packing control applications", I have
this additional wish for "r"="float16" (but you could skip this, I would not
get a nerve attack then ...).

With codec's I assume you mean something like JPEG or even MP3 compression /
decompression ... but this to my opinione really should is FAR behind any
intrinsic support in LUA ... if you want to do this you really need to
define a special "compressstring" function in you private C code ... and you
can do this if you want. JPEG or MP3 compression assumes the bit set of all
pixels of a picture is a giantic vector of 1M, or 10M bits, depending on
your image resolution, and then does some nice linear algebra with this
vector. But this you for sure can NOT do in some small C code ... this would
really be something REALLY big, needing for sure several 10-100kBytes of
code..., this needs to be EXTREMELY speed optimized, if it should run
reasonably fast, in any case I think this usually is also done fairly
platform-DEPENDENT (not independent, as lua...).

--
Sent from: http://lua.2524044.n2.nabble.com/Lua-l-f2524044.html