> I only need math functions to do some scriptable operations, where the script would be passed to lua as C-string.
Lua consists of the "core" language plus "standard" libraries. Bundled with it are a standalone compiler (luac.c) and a standalone interpreter (lua.c). Basic math operations are part of the language, plus there are some more in the math library, which you may or may not need. All the library code resides in *lib.c/h files, so at a minimum you can remove those and the compiler/interpreter from your build. That will eliminate some of the problems that you are facing.
The only other problematic stuff I can think of would then be the built-in loader, but that should be trivial to disable.