Re: [ANN] LuaFileSystem 1.7.0

[Eric Wing <ewmailing@...>]

>> Could someone please *name* a filesystem that they use Lua on that is
>> *not* windows or posix compliant?
>
>   I can name several---VMS, AmigaOS, MS-DOS 1.0, CP/M.  But ones that are
> still in use?  I don't know.  I want to say that iOS and Android file
> access
> is different, but I don't program those (I just hear the complaints at
> work).


Yeah, Android sucks. (iOS generally works with the POSIX APIs…most of
its issues are you need additional APIs for things like opting out of
automatic iCloud backups and so forth. Android also begs for
additional APIs too on top of its other problems, like finding the
paths for internal or external storage, availability, and permissions.
Both platforms (and Mac too) benefit from well known directory
locations for things like Preferences, Cache files, Documents,
Downloads, things that ship with your app bundle, etc.)

Quick answer for Android is there are two different file APIs you must
use depending on the situation. The POSIX stuff generally works for
internal and external storage (ignoring the extra hoop jumping about
permissions, availability, and finding out what all the paths to the
different locations are). But there is a different set of APIs for
reading files you ship with your app which are inside the .apk which
is basically a zip file. To complicate matters, the APIs are built
around Java and require a “God object”, a Context class, that comes
from the application and must be passed as a parameter to use these
APIs. There is no standardized way to get this Context “God” object,
which makes authoring libraries a PITA. Additionally, since the
Context object is a Java object, you must deal with Java JNI and you
also have to be able to get the JNIEnv* pointer which is like a second
God object you have to deal with. And like before, there is no
standardized way to get this either. (And the JNIEnv* cannot be shared
across threads and each thread must use a different thread local
instance of it, so you have to be extremely aware of where your
JNIEnv* instance comes from and which thread your Lua runtime exists
in.

That said, you can kind of wrap it in cross-platform wrappers (I’ve
done it before), but you need extra parameters or API for Android to
set/pass the Context object and JNIEnv*, either by passing it
explicitly, or an extra API that can store the Context for the
duration of of your life-cycle so you can pass it in on behalf of the
public API transparently to the caller.


> But in general, you are right, POSIX and Windows file systems cover
> probably 99.44% of all computer use today.

260 million PCs were sold in 2016
1.5 billion smart phones were sold in 2016


> On the other hand, a desire to continue sticking to portable code will leave LuaFileSystem or any sort of file I/O library with the usual limitations on Windows.
> Bypassing legacy library calls will get us a proper feature set. UTF-8 filenames. Bypass the MAX_PATH limitation.


As I understand it, the 260 MAX_PATH limitation on Windows can be
circumvented by doing all the following things:
1) Convert ANSI/UTF8 strings to Window’s native “wide” strings and use
all the variants of the Windows native file APIs (not ANSI/POSIX APIs)
that have the ‘W” suffix (instead of ‘A’)
2) Prefix all paths with \\?\ that you feed to these APIs. Note: This
only works for absolute paths, and not relative paths.
3) Make sure you aren’t limiting any of your own strings to the length
of MAX_PATH

In Windows 10, there now claim to be ways you can skip doing the \\?\
if you are doing (1) and (2) above.
A) Set a registry entry to disable the MAX_PATH
or
B) Set the option in an application manifest you compile into your app.


-Eric