RuntimeIOException: Parser is terminated when doing lots of requests with Connection: Keep-Alive #12356
Description
Jetty version(s)
12.0.13
Jetty Environment
core
Java version/vendor (use: java -version)
openjdk 21.0.2 2024-01-16 LTS
OS type/version
Windows 11
Description
When running Apache HTTP's ab tool the -k option (keep the connection alive), after a couple of requests (usually some hundred), ab logs apr_socket_recv: Software caused connection abort (113). When enabling debug logging in Jetty you can see these exceptions:
2024-10-09T13:39:48.325+02:00 DEBUG 8208 --- [qtp274426173-59] o.e.j.server.internal.HttpConnection : caught exception HttpConnection@16c7d92b::SocketChannelEndPoint@384e5d4f[{l=/[0:0:0:0:0:0:0:1]:8080,r=/[0:0:0:0:0:0:0:1]:55422,OSHUT,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}]->[HttpConnection@16c7d92b[p=HttpParser{s=CLOSE,0 of -1},g=HttpGenerator@6a092a24{s=START}]=>HttpChannelState@20e7b60f{handling=null, handled=false, send=SENDING, completed=false, request=null}] HttpChannelState@20e7b60f{handling=null, handled=false, send=SENDING, completed=false, request=null}
org.eclipse.jetty.io.RuntimeIOException: Parser is terminated
at org.eclipse.jetty.server.internal.HttpConnection.parseRequestBuffer(HttpConnection.java:561) ~[jetty-server-12.0.13.jar:12.0.13]
at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:386) ~[jetty-server-12.0.13.jar:12.0.13]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) ~[jetty-io-12.0.13.jar:12.0.13]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) ~[jetty-io-12.0.13.jar:12.0.13]
at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) ~[jetty-io-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478) ~[jetty-util-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441) ~[jetty-util-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) ~[jetty-util-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) ~[jetty-util-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) ~[jetty-util-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979) ~[jetty-util-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209) ~[jetty-util-12.0.13.jar:12.0.13]
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164) ~[jetty-util-12.0.13.jar:12.0.13]
at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]
How to reproduce?
I attached a reproducer Java application with spring boot and jetty.
reproducer.zip (UNSAFE, DO NOT DOWNLOAD OR USE AS-IS)
- Start it with
./gradlew bootRun - Run ab with
ab -n 100000 -k "http://localhost:8080/" - Wait for some time: on my machine a couple of hundred or thousand requests are successful