Re: xml/rapidxml UTF-8 validity assurance (was: [ANN] lub, lut, xml, yaml, dub and osc for Lua 5.3)

[Javier Guerra Giraldez <javier@...>]

On Mon, May 11, 2015 at 3:29 PM, Tim Hill <drtimhill@gmail.com> wrote:
> The problem is UTF-8 *should* be used to decode: UTF-8 -> codepoint array.
> Instead its (shudder) often used to decode UTF-8 -> UTF-16 -> (byte-swap
> based on BOM) -> codepoint array.
>
> It’s one reason I detest Unicode.


unfortunately it's the only workable solution for text intended to be
used by humans.

but here the culprit isn't Unicode, but UTF-16, which is a totally
absurd idea. it's because of UTF-16 that we got the BOM, and because
of brain-dead UTF-16-centric developers, that the BOM became valid for
UTF-8.  Initially it wasn't valid, then it was 'unrecommended', now
it's ok, and some software require it.

-- 
Javier