Subject: Re: [Boost-users] Inconsistent unicode encoding between boost andwxonmac osx
From: Peter Dimov (pdimov_at_[hidden])
Date: 2010-02-13 18:41:19
Sachin Garg wrote:
> Peter, I am guessing that the problem is not the OSX locale which
> could be UTF-8 or UTF-16 or X (I am calling this 'source' encoding).
The encoding of OS X's path names is always UTF-8. It does not depend on the
OS locale.
> Problem *seems* to be that boost converts that X into UTF16 (or UTF32)
> for storing it in wpath and that is using a normalization which is
> incompatible with normalization used by wx for its UTF16 (or 32).
I don't think that this has anything to do with normalization. By looking at
the source, Filesystem, by default, uses the global locale to convert betwen
narrow and wide paths, and the default locale in your case seems to perform
no conversion. That is, it just takes every 8 bit char and converts it to a
32 bit wchar_t. This, of course, doesn't work when the source is UTF-8,
regardless of its normalization.