Subject: Re: [boost] [review] Review of Nowide (Unicode) starts today
From: Peter Dimov (lists_at_[hidden])
Date: 2017-06-16 12:40:31
Artyom Beilis wrote:
> Actually it is what exactly you should do. Once you turn on **nowide
> integration** with boost filesystem
>
> path::string does not perform any conversions on POSIX platforms.
>
> And you can safely concatenate two different strings and valid file will
> be created. Even if dir is in ISO 8859-1 and file in UTF-8. The file will
> be valid even if not representable in any encoding.
"POSIX" is not correct here. File names under Mac OS X are UTF-8. Always
UTF-8, not merely by convention, and no mater what locale is set. (They're
stored in UTF-16 in the filesystem, just like in NTFS. Unlike NTFS, it's
_valid_ UTF-16.)
For encoding-neutral Unix-es such as Linux and Solaris, the above is
correct.
In any case, a library that attempts to reencode from the current locale to
UTF-8 in the narrow case would be useless. (If the locale isn't UTF-8,
nothing will work anyway, the library trying to handle this case would just
make things worse.) This is well established empirically by now.