lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

On 4 Dec 2006, at 18:36, Roberto Ierusalimschy wrote:

It depends on whether you want to use the encoding specified by the
current locale, or always use UTF-8.  The former is a more general
solution and is probably preferred on Unix; GNU/Linux distributions are
moving toward UTF-8 anyway.  However, it's problematic on Windows;
someone please correct me if I'm wrong, but I believe that UTF-8 is
never (or rarely) the encoding associated with the system locale on
Windows.  So if you always want to use UTF-8, it's probably better to
use a hand-written converter.

This is actually part of my question :) I guess I would prefer to use
the current locale. But I know nothing about other multibyte encodings,
and so I have no idea whether my code would work for them. For
instance, may I assume that any 0 ends the string?


What if the
encoding is state dependent?

Yes.  0 still ends the string.

(It seems a nightmare to handle shift
states when doing backtracking and the like...)


Actually dealing with shift-state dependent multi-byte encodings in a portable way in C makes the infinite horrors of Unicode and UTF-8 seem very attractive.