Skip to content

When uploading, checking 'Expect' header results in "TypeError: sequence item 0: expected str instance, bytes found" #3111

@genman

Description

@genman

Describe the bug

Uploading a binary file to S3 instance (not AWS). This error is occurring:

Traceback (most recent call last):
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/httpsession.py", line 464, in send
urllib_response = conn.urlopen(
File "/miniconda/envs/iris/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
File "/miniconda/envs/iris/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
conn.request(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/awsrequest.py", line 91, in request
if headers.get('Expect', b'') == b'100-continue':

^^^ Why is this b''?

File "/miniconda/envs/iris/lib/python3.10/_collections_abc.py", line 824, in get
return self[key]
File "/miniconda/envs/iris/lib/python3.10/site-packages/urllib3/_collections.py", line 261, in getitem
return ", ".join(val[1:])
TypeError: sequence item 0: expected str instance, bytes found

^^^ I'm not sure the value of 'val' but

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...
File "/miniconda/envs/iris/lib/python3.10/site-packages/boto3/s3/inject.py", line 145, in upload_file
return transfer.upload_file(
File "/miniconda/envs/iris/lib/python3.10/site-packages/boto3/s3/transfer.py", line 371, in upload_file
future.result()
File "/miniconda/envs/iris/lib/python3.10/site-packages/s3transfer/futures.py", line 103, in result
return self._coordinator.result()
File "/miniconda/envs/iris/lib/python3.10/site-packages/s3transfer/futures.py", line 266, in result
raise self._exception
File "/miniconda/envs/iris/lib/python3.10/site-packages/s3transfer/tasks.py", line 139, in call
return self._execute_main(kwargs)
File "/miniconda/envs/iris/lib/python3.10/site-packages/s3transfer/tasks.py", line 162, in _execute_main
return_value = self._main(**kwargs)
File "/miniconda/envs/iris/lib/python3.10/site-packages/s3transfer/upload.py", line 764, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/client.py", line 553, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/client.py", line 989, in _make_api_call
http, parsed_response = self._make_request(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/client.py", line 1015, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/endpoint.py", line 119, in make_request
return self._send_request(request_dict, operation_model)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/endpoint.py", line 202, in _send_request
while self._needs_retry(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/endpoint.py", line 354, in _needs_retry
responses = self._event_emitter.emit(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/hooks.py", line 412, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/hooks.py", line 256, in emit
return self._emit(event_name, kwargs)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/hooks.py", line 239, in _emit
response = handler(**kwargs)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/retryhandler.py", line 207, in call
if self._checker(**checker_kwargs):
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/retryhandler.py", line 284, in call
should_retry = self._should_retry(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/retryhandler.py", line 307, in _should_retry
return self._checker(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/retryhandler.py", line 363, in call
checker_response = checker(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/retryhandler.py", line 247, in call
return self._check_caught_exception(
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception
raise caught_exception
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/endpoint.py", line 281, in _do_get_response
http_response = self._send(request)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/endpoint.py", line 377, in _send
return self.http_session.send(request)
File "/miniconda/envs/iris/lib/python3.10/site-packages/botocore/httpsession.py", line 509, in send
raise HTTPClientError(error=e)
botocore.exceptions.HTTPClientError: An HTTP Client raised an unhandled exception: sequence item 0: expected str instance, bytes found

Expected Behavior

Upload to work, no exception

Current Behavior

Exception as shown

Reproduction Steps

Likely requires S3 server that's not AWS

Possible Solution

Why are headers checked like this? Maybe use str types not byte?

Additional Information/Context

No response

SDK version used

1.34.24

Environment details (OS name and version, etc.)

Linux, Python 3.10

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a confirmed bug.dependenciesThis issue is a problem in a dependency.third-party

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions