This topic might have been discussed before.
We have proprietary lua code (along with C/C++ binaries) made available as a self-downloadable package (by our customers) and we need to keep it hidden for various reasons (security, IP protection, etc.).
What is the recommended mechanism in lua for this?
There isn't one. That's an application-level thing. Use whatever scheme you would use to protect any other data, and just unpack it right before you lua_load it.
It's been a very long time since I needed to do this, but at the time I just went for basic anti-casual-snooping protection -- I packed it all into a zip file, then XOR'ed the whole file with a constant byte, then assigned a custom read callback into an unzip library that undid the XOR on the fly. I have no idea if anyone ever even bothered to try to crack it, but it worked well enough to avoid making virus scanners think it's an archive file that needs analyzed.