Skip to content

utf8::utf16to8 and utf8::unchecked::utf16to8 produce different output #111

@GPUex

Description

@GPUex

As title mentions, checked and unchecked variations of utf16to8 produce different output.
Simple test case:

#include <iostream>
#include <format>
#include "utf8.h"

std::string u16_to_u8_unchecked(const std::u16string& u16)
{
	std::string str;
	utf8::unchecked::utf16to8(u16.begin(), u16.end(), std::back_inserter(str));
	return str;
}

int main()
{
	std::u16string u16 = u"Simple test!";

	std::string u8_checked = utf8::utf16to8(u16);
	std::string u8_unchecked = u16_to_u8_unchecked(u16);

	std::cout << std::format("{}\n{}\n", u8_checked, u8_unchecked);
	std::cout << std::format("{} {} {}\n", u8_checked == u8_unchecked, u8_checked.size(), u8_unchecked.size());

	return 0;
}

Produces this output:

Simple test!
Simple test
false 12 11

Library version 4.0
Platform:
Windows 11 (10.0.22621.2428)
Visual Studio 2022 (17.7.6) (compiler version 19.37.32825.0)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions