Skip to content

Cannot create object/file - Openstack Swift Keystone 3.0 / HTTP response code 500 was returned, slowing down #357

@vguilleaume

Description

@vguilleaume

I'm facing with an issue with S3FS -> S3PROXY connected to an OpenStack Swift - Keystone 3.0

Connection is working as expected.
We see that the system hang when creating file in container swift.

The system create object but do not allow to save it. System can list, read existing contents , but the creating through S3FS - S3proxy do not work. Furthermore , when we tried to try to create one, object is defined, but stay empty and from there we cannot anymore list the object in the container.

We see error message such as :
HTTP response code 500 was returned, slowing down

We can reproduce the problem

Launch the swift proxy ( s3proxy ) with the connector

s3proxy --properties /etc/s3proxy/swiftconnector

Run the mount access

s3fs -f -o url=http://127.0.0.1:8000 -o passwd_file=/etc/passwd-s3fs/swift -o use_path_request_style my_container /mnt/Pool/7/my_container/switchcontainer2/

When creating a new file, i receive a slowing down error message :
at org.gaul.s3proxy.S3ProxyHandlerJetty.handle(S3ProxyHandlerJetty.java:76) at org.gaul.shaded.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.gaul.shaded.org.eclipse.jetty.server.Server.handle(Server.java:499) at org.gaul.shaded.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.gaul.shaded.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.gaul.shaded.org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.gaul.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.gaul.shaded.org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:748) 2021-04-26T09:26:36.772Z [INF] curl.cpp:RequestPerform(2347): HTTP response code 500 was returned, slowing down

We analyste the difference between the 2 files, first one good → second one not good

(my_env) [root@test ~]# openstack object show my_container my_file.txt

+----------------+---------------------------------------+
| Field | Value |
+----------------+---------------------------------------+
| account | AUTH_481b9914945142b5b404dd46b70ae1e4 |
| container | my_container |
| content-length | 18 |
| content-type | text/plain; charset=UTF-8 |
| etag | f15e03e6bdcc396aa00e7a76c08d6e54 |
| last-modified | Mon, 26 Apr 2021 07:52:54 GMT |
| object | my_file.txt |
+----------------+---------------------------------------+
(my_env) [root@datamoer-test ~]# openstack object show my_container my_file2.txt
+---------------+--------------------------------------------------------------------------------------------+
| Field | Value |
+---------------+--------------------------------------------------------------------------------------------+
| account | AUTH_481b9914945142b5b404dd46b70ae1e4 |
| container | my_container |
| content-type | text/plain; charset=UTF-8 |
| etag | d41d8cd98f00b204e9800998ecf8427e |
| last-modified | Mon, 26 Apr 2021 09:21:29 GMT |
| object | my_file2.txt |
| properties | Atime='1619428887', Ctime='1619428887', Gid='0', Mode='33188', Mtime='1619428887', Uid='0' |
+---------------+--------------------------------------------------------------------------------------------+

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions