插入窄字符串到 std::basic_ostream<wchar_t>

Question

根据cppref, there is an operator << overload for std::basic_ostream<wchar_t> that accepts const char*. It seems that the convert operation simply widens each char into a wchar_t. That is, the number of wide characters converted (inserted) is equal to the number of narrow characters. So here comes a problem. The narrow character string may be encoding international characters, say Chinese characters using GB2312。进一步假设 sizeof(wchar_t) 是 2 并使用 UTF16 编码。那么这种天真的字符转换方法应该如何工作？

Answer 1

我刚刚签到 Visual Studio 2015，你是对的。 char 仅加宽为 wchar_t，没有任何转换。在我看来，您必须自己将窄字符串转换为宽字符串。有几种方法可以做到这一点，其中一些已经被建议了。

在这里我建议你可以使用纯 C++ 工具来完成它，假设你的 C++ 编译器和标准库足够完整（Visual Studio，或者 GCC on Linux（而且只有那里））：

void clear_mbstate (std::mbstate_t & mbs);

void
towstring_internal (std::wstring & outstr, const char * src, std::size_t size,
    std::locale const & loc)
{
    if (size == 0)
    {
        outstr.clear ();
        return;
    }

    typedef std::codecvt<wchar_t, char, std::mbstate_t> CodeCvt;
    const CodeCvt & cdcvt = std::use_facet<CodeCvt>(loc);
    std::mbstate_t state;
    clear_mbstate (state);

    char const * from_first = src;
    std::size_t const from_size = size;
    char const * const from_last = from_first + from_size;
    char const * from_next = from_first;

    std::vector<wchar_t> dest (from_size);

    wchar_t * to_first = &dest.front ();
    std::size_t to_size = dest.size ();
    wchar_t * to_last = to_first + to_size;
    wchar_t * to_next = to_first;

    CodeCvt::result result;
    std::size_t converted = 0;
    while (true)
    {
        result = cdcvt.in (
            state, from_first, from_last,
            from_next, to_first, to_last,
            to_next);
        // XXX: Even if only half of the input has been converted the
        // in() method returns CodeCvt::ok. I think it should return
        // CodeCvt::partial.
        if ((result == CodeCvt::partial || result == CodeCvt::ok)
            && from_next != from_last)
        {
            to_size = dest.size () * 2;
            dest.resize (to_size);
            converted = to_next - to_first;
            to_first = &dest.front ();
            to_last = to_first + to_size;
            to_next = to_first + converted;
            continue;
        }
        else if (result == CodeCvt::ok && from_next == from_last)
            break;
        else if (result == CodeCvt::error
            && to_next != to_last && from_next != from_last)
        {
            clear_mbstate (state);
            ++from_next;
            from_first = from_next;
            *to_next = L'?';
            ++to_next;
            to_first = to_next;
        }
        else
            break;
    }
    converted = to_next - &dest[0];

    outstr.assign (dest.begin (), dest.begin () + converted);
}

void
clear_mbstate (std::mbstate_t & mbs)
{
    // Initialize/clear mbstate_t type.
    // XXX: This is just a hack that works. The shape of mbstate_t varies
    // from single unsigned to char[128]. Without some sort of initialization
    // the codecvt::in/out methods randomly fail because the initial state is
    // random/invalid.
    std::memset (&mbs, 0, sizeof (std::mbstate_t));
}

这个函数是 log4cplus 库的一部分，它可以工作。它使用 codecvt 方面进行转换。你必须给它适当地设置 locale.

Visual studio 可能无法正确设置 GB2312 的语言环境。您可能必须使用 _setmbcp() 才能使其正常工作。有关详细信息，请参阅“double byte character sequence conversion issue in Visual Studio 2015”。

插入窄字符串到 std::basic_ostream<wchar_t>

Inserting narrow character string to std::basic_ostream<wchar_t>

c++

locale

internationalization

c++-standard-library