Subject: Re: [boost] Review Request: Boost.Locale
From: Robert Ramey (ramey_at_[hidden])
Date: 2010-05-24 04:47:41
Artyom wrote:
>> One thing in particular that I was interested in is(are)
>> codecvt
>> facets.
>> I didn't any thing on this. Why is
>> that?
>
> Take a look on http://www.cplusplus.com/reference/std/locale/codecvt/
>
> They allow you to imbue special charset to fstream and automatically
> translate wide characters to normal encoding like UTF-8 or ISO-8859-8.
>
>
>> Is this a separate
>> subject or is that you believe they're not useful.
>
> Theoretically they are very useful.
>
> For example:
>
> std::wofstream fs;
> fs.imbue(std::locale("he_IL.UTF-8"));
> fs.open("file.txt");
> fs << L"????"!
>
> Would print UTF-8 output.
I'm familiar with the codecvt facet as several of them are used
with the serialization library.
>
> But...
>
> - Many compilers/standard libraries do not implement locales at all.
> (GCC under Windows and Solaris, STL Port library)
> - Support of locales and encoding is strictly limited to OS
> configuration. So on some host the above example would work on other
> it would throw invalid locale error.
> - Some compilers/OSes do not support UTF-8 encodings (MSVC) so
> you can't create UTF-8 locale at all.
> - Locales name are platform depended. For example under Windows
> you need Hebrew_Israel.1255 locale and under Linux he_IL.ISO-8859-8
> (and BTW 1255!=iso-8859-8)
I think this is a separate issue than codecvt facet. I've found them to work
with all C++ implemenations that boost uses.
> So Boost.Locale reimplements standard codecvt facet to make this work
> on any platform.
I didn't seen anything in the documentation about that.
>
> However there is still a limitation when working with 2 byte
> characters (ie char16_t or wchar_t under windows) as Boost.Locale
> would work correctly only with UCS-2
> But this is actually C++ standard's limitation.
The reason I ask is that I often see things requested on the list
which I think could be better implemented as codecvt facets.
Also, it seemed to me that a large part of the iostreams library
could have been implemented more efficiently wiht codecvt
facets. Admitidly, it's somewhat unobvious how to make the
best use of this - (needs another library and documentation of course)
but I'm surprised that don't seem to be mentioned at all
in the documentation.
Robert Ramey