Re: string dollar literals (for luamacro or tokenf)

[Tony Finch <dot@...>]

Jay Carlson <nop@nop.com> wrote:

Thanks for your interesting response.

> You've already lost when the problem is stated in terms of escaping,
> at least as a task other than specialist library code does. Types are
> the problem. Even the duck typing meme does not work on strings. It's
> just as bad as BCPL.

Right, that's exactly what I was hinting at by mentioning stringly typed
systems. (Also, Tcl.)

You might design a system with more precisely typed data, where the type
of a string corresponds to its formal language, i.e. the syntax of the
string. So strings aren't just strings, they are SQL strings or JSON
strings or passwords, etc. Then each slot in a template needs to know the
type of string that it accepts. So a SQL query template might have a slot
that only accepts a quoted SQL string literal. You can't just interpolate
a username into that slot, you need to do a type conversion, and part of
that conversion includes putting quotes around the string and escaping
metacharacters correctly.

There have been at least two Google templating libraries that sort of
works along these lines, but they use parsers with a lot of built-in
knowledge of web languages to derive the typing requirements.
http://googleonlinesecurity.blogspot.com/2009/03/reducing-xss-by-way-of-automatic.html
http://google-caja.googlecode.com/svn/changes/mikesamuel/string-interpolation-29-Jan-2008/trunk/src/js/com/google/caja/interp/index.html
And they are rather heavy-weight, so difficult to imitate.

A more principled approach is to parse everything as it comes in off the
wire into a tree-structured internal representation. Instead of
interpolating strings into templates you graft tree nodes into slots.
You then serialize the parse tree back into its external representation
when sending it out, and the serializer will naturally either do the right
thing or fail clearly.

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
Faeroes: South or southeast 5 to 7, occasionally gale 8 later. Mainly rough.
Rain. Good, occasionally poor.