Do you really need to store this data in json ? Could something like msgpack (there are Lua libraries available) suit you better?
Also, you may want to try the other way. If the binary data is withing a given set of easily identifiable stuff (like, image files with some kind of header) you can try to identify those first, then fallback to scanning the string and check for a valid string.
The page on the wiki on json libraries lists many of them and also how they cope with valid and invalid stuff:
Regards,
Ignacio