Introducing LuaZDF - Lua Zero Dependency Functions

[Alexander Schulz <aiqpost@...>]

Hi,

I did developed/collected some util functions for Lua.
You can check out the functions at https://luazdf.aiq.dk/ or on https://github.com/aiq/luazdf.

Cheers,
Alexander

HTML version: https://blog.aiq.dk/introducing-luazdf.html

Introducing LuaZDF
==================

[abstract]
LuaZDF - Lua Zero Dependency Functions 

== Problem

For the development of my little Lua library basexx<<1>> I needed a simple
utility function that splits a string into an array table with substrings that
have the same length.
This function is not part of the Lua standard libraries<<2>>, a quick search in
the LuaRocks<<3>> repository was also unsuccessful.
Other languages offer this kind of function in their standard library<<4>>.
In my case, I ended up implementing the function by myself.
This specific function is not rocket science by itself, but it is always easy to
create bugs when implementing software, so I would prefer an implemented and
tested function for this case.

Now it is easy to imagine that I need this function for other projects, so the
question is how to share this and other similar functions.
Therefore I introduce LuaZDF and will now explain the main design decision that
I made in developing it.

== Common Approaches

There are three common approaches I know to collect and distribute simple
utility functions.
I will describe them with a few words.

=== Big Module

One approach is to collect this kind of utility functions in a single module and
to include this module in the project.
The module would collect all kind of functions: io, string, color, etc - like
Penlight<<5>> for example.
For me, the size of this imaginary module is the biggest disadvantage of this
approach, as I would need this module for just one function, but would demand 
that the rest does not bother me.
Everyone who worked with big environments and libraries knows the hassle that
follows by including big modules into a project.
You will start to depend on the decision made by the library, like deep class
structures and conventions.

=== Domain Module

You can reduce the size of one big module by creating single modules for each
domain (io, string, color, etc), but they will still include a bunch of
functions that are not required.
My little Lua library basexx itself is this kind of module.
A quick look at the projects<<6>> that are using basexx shows that just one or
two functions from the module are actually used.
The basexx module currently contains 14 functions, which means that around 90%
of the functions are not required for those projects, thus adding unnecessary
amounts of data to the project.
Also I would have to add another module dependency for my imaginary string
library that contains my util function.
This leads to even more unused code and thus more unnecessary resources.

=== Single Function Module

Considering the downsides that large amounts of unnecessary code have on your
project, the approach to create a single module for each function seems
reasonable.
The npm package manager<<7>> contains a lot modules that match this approach.
The dependency tree that this approach generates can of course lead to new
problems<<8>>.
In generall, I am in favour of this approach, as you can specifically include
those dependencies that you really need.
The main downside of this is that you end up with a lot of dependencies that
need to be managed, one for each function.

== Zero Dependency Functions (ZDF)

For LuaZDF, the available functions will be implemented separately in single
files for each function.
LuaZDF is not library like the common approaches, it is more a function
registry.
This approach seems similar to the "Single Function Module" approach but has one
major difference: you do not have to handle dependencies, as every function/file
contains all necessary source code for it to function on its own.
LuaZDF follows the principle that is best described with the following Go
Proverb<<9>>:
....
A little copying is better than a little dependency.
....
This means that all functions work out of the box when you copy them into your
project.
I need to tell you one restriction in the second sentence, that is to say that
not all dependencies
are expendable.
For example, it is hard in Lua to work on the file system without using 
LuaFileSystem<<10>>.
But all functions inside the ZDF registry are independent and have zero
dependencies among themselves.

Let me show you with an example what that means.
The usefull function<<11>> _dayofweek_ requires the function _isleapyear_ to
work.
The DRY principle<<12>> would automatically lead to two seperate functions.
That is not wrong, but it leads to abstraction that hides functionality and adds
dependency.

With the ZDF approach I try to offer/collect all kind of useful functions that
are easy to add into a project and have a small footprint regarding their
dependencies and size.

== Coding vs. Managing

You may still question yourself: "Why do you not just add single function
modules to LuaRocks?"
That was the first question from a friend, when I told him about the ZDF
approach.
For me the main problem with every package manager is the task of managing all
the dependencies.
I love to code, but everything that has to do with managing is a horror for me.

I talk about both sides here, the creation and use of packages.
The convention over configuration<<13>> paradigm reduces a lot of work, but I
have problems with creating the packages.
LuaRocks for example already has a
simple and easy way to create a package<<14>>, but that does not help me much.
I always struggle with things that feel like configuration.
For me, creating and updating the rockspec file<<15>> is the part with the
lowest level of joy.

The main problems on using packages often are the dependencies and their
management.
For example almost all package managers force you to define a required version
for a package.
My experience is that in most cases the packages are just defining the minimum
version and ignore a maximum version where the code compiles or runs correctly.
I unterstand the motivation but it breaks my code so often - it really hurts.

On the other side, I like approaches like stb<<16>> where you can just copy the
code into your project, and have nothing to manage afterwards.
Also, the code is viewable up front and is not hidden within a package
structure.
I don't know how to discribe it better as that "you actually see the code".
You also have the responsibility to include new versions of the used code to
your project, as there is no automatic management of getting newer versions - I
hope that the simple kind of the functions within LuaZDF do not require many
updates.

== Closing Words

I tried to give you a rough understanding about the ideas and decisions behind
LuaZDF.
If things are still unclear, check out the website<<17>> for more information or
write a question on reddit<<18>>.

== Sources
[bibliography]
- [[[1]]] https://github.com/aiq/basexx
- [[[2]]] https://www.lua.org/manual/5.3/manual.html#6
- [[[3]]] https://luarocks.org/
- [[[4]]] http://php.net/manual/en/function.str-split.php
- [[[5]]] http://stevedonovan.github.io/Penlight/api/index.html
- [[[6]]] https://luarocks.org/modules/aiq/basexx
- [[[7]]] https://www.npmjs.com/
- [[[8]]] http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how-to-program/
- [[[9]]] https://go-proverbs.github.io/
- [[[10]]] https://keplerproject.github.io/luafilesystem/
- [[[11]]] http://onemanmmo.com/index.php?cmd=newsitem&comment=news.1.392.0 
- [[[12]]] https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
- [[[13]]] https://github.com/luarocks/luarocks/wiki/Creating-a-rock
- [[[14]]] https://github.com/aiq/basexx/blob/master/dist/basexx-0.4.0-1.rockspec
- [[[15]]] https://en.wikipedia.org/wiki/Convention_over_configuration
- [[[16]]] https://github.com/nothings/stb
- [[[17]]] https://luazdf.aiq.dk/
- [[[18]]] https://www.reddit.com/r/LuaZDF/