[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: question about Unicode
- From: David Jones <drj@...>
- Date: Thu, 7 Dec 2006 12:52:49 +0000
On 4 Dec 2006, at 18:36, Roberto Ierusalimschy wrote:
It depends on whether you want to use the encoding specified by the
current locale, or always use UTF-8. The former is a more general
solution and is probably preferred on Unix; GNU/Linux
moving toward UTF-8 anyway. However, it's problematic on Windows;
someone please correct me if I'm wrong, but I believe that UTF-8 is
never (or rarely) the encoding associated with the system locale on
Windows. So if you always want to use UTF-8, it's probably better to
use a hand-written converter.
This is actually part of my question :) I guess I would prefer to use
the current locale. But I know nothing about other multibyte
and so I have no idea whether my code would work for them. For
instance, may I assume that any 0 ends the string?
What if the
encoding is state dependent?
Yes. 0 still ends the string.
(It seems a nightmare to handle shift
states when doing backtracking and the like...)
Actually dealing with shift-state dependent multi-byte encodings in a
portable way in C makes the infinite horrors of Unicode and UTF-8
seem very attractive.