Python 3.6 on Windows click.echo() truncates output click v6.7, unable to duplicate in test #816
Description
There appears to be a limit of ~16 383 character output when using click.echo() on a windows environment (Windows 7 64-bit) in python 3.6.0. Similar behavior is not present using python 2.7.11. This behavior seems to include escape characters to varying degrees. I've checked it using the following:
import click
def string_construct(string):
string_list = []
for i in range(17000):
string_list.append(string)
return string_list
@click.command()
@click.option('--first', '-1', default=False, is_flag=True)
@click.option('--second', '-2', default=False, is_flag=True)
@click.option('--third', '-3', default=False, is_flag=True)
@click.option('--fourth', '-4', default=False, is_flag=True)
def main(first, second, third, fourth):
if first:
s = ''.join(string_construct('t'))
print(len(s))
click.echo(s)
elif second:
s = ''.join(string_construct('\tt'))
print(len(s))
click.echo(s)
elif third:
n = 'A somewhat longer string to constrain console output\n'
s = ''.join(string_construct(n))
print(len(s))
click.echo(s)
elif fourth:
s = ''.join(string_construct(' '))
print(len(s))
click.echo(s)
list1:
- output: 16 383 characters
- no escaped
list2:
- output: 8 191 characters
- with tabs as 1 character: 16 382
list3:
- output: 15 799 characters (spaces included)
- with newline as 1 character: 16 092
- with newline as 2 characters: 16 385
list4:
- output: 16 235 characters
I copied the output over to word to check character counts, which is probably were the discrepancies come from. The output of len(s) is always what you would expect. I've started working on a patch in _compat.py in an elif WIN: block below the following block:
# The io module is a place where the Python 3 text behavior
# was forced upon Python 2, so we need to unbreak
# it to look like Python 2.
if PY2:
def write(self, x):
if isinstance(x, str) or is_bytes(x):
try:
self.flush()
except Exception:
pass
return self.buffer.write(str(x))
return io.TextIOWrapper.write(self, x)
def writelines(self, lines):
for line in lines:
self.write(line)
Right now I have a working patch using the following code (extending the above if):
elif WIN:
def write(self, x):
MAX_LEN = 16000
message_len = len(x)
if message_len > MAX_LEN:
self.write(x[0:MAX_LEN])
return self.write(x[MAX_LEN:message_len])
else:
return io.TextIOWrapper.write(self, x)
I've tried to write a valid test, the trouble is that runner and capsys.readouterr() are both showing the output at the correct length. I've double checked and my patch works to correct the error, I just can't duplicate the error in the test environment.