Skip to content

UnicodeDecodeError on failing assert if non-ASCII characters in __repr__ #3999

@dtomas

Description

@dtomas

If a UTF-8 encoded string is returned in the __repr__ method of a class, a UnicodeDecodeError is raised if an assert fails while checking any unicode attribute on an object of that class (even ASCII-decodable ones). Non-unicode attributes (including byte strings) work as expected.

# -*- coding: utf-8 -*-

class A(object):
    name = u"xyzä"
    unicode_attr = u"a"
    ascii_str_attr = "b"
    utf8_str_attr = u"ä".encode("utf-8")

    def __repr__(self):
        return self.name.encode("utf-8")

def test_unexpected_unicodedecodeerror():
    a = A()
    assert not a.name

def test_unexpected_unicodedecodeerror2():
    a = A()
    assert not a.unicode_attr

def test_expected_assertionerror():
    a = A()
    assert not a.ascii_str_attr

def test_expected_assertionerror2():
    a = A()
    assert not a.utf8_str_attr
============================= test session starts ==============================
platform darwin -- Python 2.7.15, pytest-3.8.0, py-1.6.0, pluggy-0.7.1
rootdir: /Users/dtomas/work, inifile:
collected 4 items                                                              

test_unicode.py FFFF                                                     [100%]

=================================== FAILURES ===================================
______________________ test_unexpected_unicodedecodeerror ______________________

    def test_unexpected_unicodedecodeerror():
        a = A()
>       assert not a.name
E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)

test_unicode.py:14: UnicodeDecodeError
_____________________ test_unexpected_unicodedecodeerror2 ______________________

    def test_unexpected_unicodedecodeerror2():
        a = A()
>       assert not a.unicode_attr
E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)

test_unicode.py:18: UnicodeDecodeError
_________________________ test_expected_assertionerror _________________________

    def test_expected_assertionerror():
        a = A()
>       assert not a.ascii_str_attr
E       AssertionError: assert not 'b'
E        +  where 'b' = xyzä.ascii_str_attr

test_unicode.py:22: AssertionError
________________________ test_expected_assertionerror2 _________________________

    def test_expected_assertionerror2():
        a = A()
>       assert not a.utf8_str_attr
E       AssertionError: assert not '\xc3\xa4'
E        +  where '\xc3\xa4' = xyzä.utf8_str_attr

test_unicode.py:26: AssertionError
=========================== 4 failed in 0.04 seconds ===========================
$ pip list
Package        Version
-------------- -------
atomicwrites   1.2.1  
attrs          18.2.0 
funcsigs       1.0.2  
more-itertools 4.3.0  
pathlib2       2.3.2  
pip            18.0   
pluggy         0.7.1  
py             1.6.0  
pytest         3.8.0  
scandir        1.9.0  
setuptools     40.4.1 
six            1.11.0 
wheel          0.31.1 

Might be related to #877 and #678.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugproblem that needs to be addressed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions