Skip to content

Conversation

avahahn
Copy link

@avahahn avahahn commented Aug 22, 2024

No description provided.

ac000 added 2 commits August 22, 2024 22:30
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
@avahahn avahahn changed the base branch from master to 1.33-prep August 22, 2024 22:33
Signed-off-by: Ava Hahn <a.hahn@f5.com>
@ac000
Copy link
Owner

ac000 commented Aug 22, 2024

Thanks, think I'll just fold it into the changes.xml commit...

@ac000
Copy link
Owner

ac000 commented Aug 23, 2024

Change was merged, thanks!

@ac000 ac000 closed this Aug 23, 2024
ac000 pushed a commit that referenced this pull request Aug 28, 2024
Otherwise, undefined behaviour will be triggered.

Can be reproduced by test/test_routing.py::test_routes_match_host_empty
with enabled UndefinedBehaviorSanitizer:

src/nxt_http_route.c:2141:17: runtime error: applying zero offset to null pointer
    #0 0x100562588 in nxt_http_route_test_rule nxt_http_route.c:2091
    #1 0x100564ed8 in nxt_http_route_handler nxt_http_route.c:1574
    #2 0x10055188c in nxt_http_request_action nxt_http_request.c:570
    #3 0x10052b1a0 in nxt_h1p_request_body_read nxt_h1proto.c:998
    nginx#4 0x100449c38 in nxt_event_engine_start nxt_event_engine.c:542
    nginx#5 0x100436828 in nxt_thread_trampoline nxt_thread.c:126
    nginx#6 0x18133e030 in _pthread_start+0x84 (libsystem_pthread.dylib:arm64e+0x7030)
    nginx#7 0x181338e38 in thread_start+0x4 (libsystem_pthread.dylib:arm64e+0x1e38)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/nxt_http_route.c:2141:17

Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
ac000 pushed a commit that referenced this pull request Aug 28, 2024
Can be reproduced by test/test_rewrite.py::test_rewrite_njs
with enabled UndefinedBehaviorSanitizer:

src/nxt_http_js.c:169:52: runtime error: applying zero offset to null pointer
    #0 0x10255b044 in nxt_http_js_ext_get_args nxt_http_js.c:169
    #1 0x102598ad0 in njs_value_property njs_value.c:1175
    #2 0x10259c2c8 in njs_vm_object_prop njs_vm.c:1398
    #3 0x102559d74 in nxt_js_call nxt_js.c:445
    nginx#4 0x1023c0da0 in nxt_tstr_query nxt_tstr.c:276
    nginx#5 0x102516ec4 in nxt_http_rewrite nxt_http_rewrite.c:56
    nginx#6 0x1024fd86c in nxt_http_request_action nxt_http_request.c:565
    nginx#7 0x1024d71b0 in nxt_h1p_request_body_read nxt_h1proto.c:998
    nginx#8 0x1023f5c48 in nxt_event_engine_start nxt_event_engine.c:542
    nginx#9 0x1023e2838 in nxt_thread_trampoline nxt_thread.c:126
    nginx#10 0x18133e030 in _pthread_start+0x84 (libsystem_pthread.dylib:arm64e+0x7030)
    nginx#11 0x181338e38 in thread_start+0x4 (libsystem_pthread.dylib:arm64e+0x1e38)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/nxt_http_js.c:169:52

Same fix was introduced in NJS:
<http://hg.nginx.org/njs/rev/4fba78789fe4>

Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
ac000 pushed a commit that referenced this pull request Aug 28, 2024
Found by UndefinedBehaviorSanitizer:

src/nxt_random.c:151:31: runtime error: left shift of 140 by 24 places cannot be represented in type 'int'
    #0 0x104f78968 in nxt_random nxt_random.c:151
    #1 0x104f58a98 in nxt_shm_open nxt_port_memory.c:377
    #2 0x10503e24c in nxt_controller_conf_send nxt_controller.c:617
    #3 0x105041154 in nxt_controller_process_request nxt_controller.c:1109
    nginx#4 0x104fcdc48 in nxt_event_engine_start nxt_event_engine.c:542
    nginx#5 0x104f27254 in main nxt_main.c:35
    nginx#6 0x180fbd0dc  (<unknown module>)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/nxt_random.c:151:31

Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
ac000 pushed a commit that referenced this pull request Aug 28, 2024
Can be reproduced by test/test_variables.py::test_variables_dynamic_arguments
with enabled UndefinedBehaviorSanitizer:

src/nxt_http_request.c:961:17: runtime error: applying zero offset to null pointer
    #0 0x1050d95a4 in nxt_http_arguments_parse nxt_http_request.c:961
    #1 0x105102bf8 in nxt_http_var_arg nxt_http_variables.c:621
    #2 0x104f95d74 in nxt_var_interpreter nxt_var.c:507
    #3 0x104f98c98 in nxt_tstr_query nxt_tstr.c:265
    nginx#4 0x1050abfd8 in nxt_router_access_log_writer nxt_router_access_log.c:194
    nginx#5 0x1050d81f4 in nxt_http_request_close_handler nxt_http_request.c:838
    nginx#6 0x104fcdc48 in nxt_event_engine_start nxt_event_engine.c:542
    nginx#7 0x104fba838 in nxt_thread_trampoline nxt_thread.c:126
    nginx#8 0x18133e030 in _pthread_start+0x84 (libsystem_pthread.dylib:arm64e+0x7030)
    nginx#9 0x181338e38 in thread_start+0x4 (libsystem_pthread.dylib:arm64e+0x1e38)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/nxt_http_request.c:961:17

Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
ac000 pushed a commit that referenced this pull request Aug 28, 2024
This issue was found with oss-fuzz.

  ==18420==WARNING: MemorySanitizer: use-of-uninitialized-value
	      #0 0x55dd798a5797 in nxt_vsprintf unit/src/nxt_sprintf.c:163:31
	      #1 0x55dd798d5bdb in nxt_conf_vldt_error unit/src/nxt_conf_validation.c:1525:11
	      #2 0x55dd798dd4cd in nxt_conf_vldt_var unit/src/nxt_conf_validation.c:1560:16
	      #3 0x55dd798dd4cd in nxt_conf_vldt_if unit/src/nxt_conf_validation.c:1592:16
	      nginx#4 0x55dd798d55f4 in nxt_conf_vldt_object unit/src/nxt_conf_validation.c:2815:23
	      nginx#5 0x55dd798d6f84 in nxt_conf_vldt_access_log unit/src/nxt_conf_validation.c:3426:11
	      nginx#6 0x55dd798d55f4 in nxt_conf_vldt_object unit/src/nxt_conf_validation.c:2815:23
	      nginx#7 0x55dd798d47bd in nxt_conf_validate unit/src/nxt_conf_validation.c:1421:11
	      nginx#8 0x55dd79871c82 in LLVMFuzzerTestOneInput unit/fuzzing/nxt_json_fuzz.c:67:5
	      nginx#9 0x55dd79770620 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:614:13
	      nginx#10 0x55dd7975adb4 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:327:6
	      nginx#11 0x55dd7976084a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:862:9
	      nginx#12 0x55dd7978cc42 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
	      nginx#13 0x7e8192213082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
	      nginx#14 0x55dd7975188d in _start

	    Uninitialized value was created by an allocation of 'error.i' in the stack frame
	      #0 0x55dd798dd42b in nxt_conf_vldt_var unit/src/nxt_conf_validation.c:1557:5
	      #1 0x55dd798dd42b in nxt_conf_vldt_if unit/src/nxt_conf_validation.c:1592:16

The issue was in nxt_tstr_test() where we create an error message with
nxt_sprintf(), where this error message is then later used with the
'%s' format specifier which expects a nul-terminated string, but by
default nxt_sprintf() doesn't nul-terminate, you must use the '%Z'
specifier to signify a '\0' at the end of the string.

Signed-off-by: Arjun <pkillarjun@protonmail.com>
Co-developed-by: Zhidao HONG <z.hong@f5.com>
Signed-off-by: Zhidao HONG <z.hong@f5.com>
Link: <https://github.com/google/oss-fuzz>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
[ Commit message/subject - Andrew ]
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
ac000 added a commit that referenced this pull request Apr 9, 2025
valgrind(1) was producing the following alerts

  ==166470== Syscall param sendmsg(msg.msg_iov[0]) points to uninitialised byte(s)
  ==166470==    at 0x4AE6514: sendmsg (sendmsg.c:28)
  ==166470==    by 0x42D86C: nxt_sendmsg (nxt_socket_msg.c:32)
  ==166470==    by 0x4FE6695: nxt_unit_sendmsg (nxt_unit.c:6013)
  ==166470==    by 0x4FEB6E2: nxt_unit_ready (nxt_unit.c:963)
  ==166470==    by 0x4FEB6E2: nxt_unit_init (nxt_unit.c:557)
  ==166470==    by 0x4FEEC56: nxt_php_start (nxt_php_sapi.c:507)
  ==166470==    by 0x426BA0: nxt_app_setup (nxt_application.c:1029)
  ==166470==    by 0x403153: nxt_process_do_start (nxt_process.c:718)
  ==166470==    by 0x4042A3: nxt_process_whoami_ok (nxt_process.c:846)
  ==166470==    by 0x407A28: nxt_port_rpc_handler (nxt_port_rpc.c:347)
  ==166470==    by 0x407E42: nxt_port_handler (nxt_port.c:184)
  ==166470==    by 0x40501B: nxt_port_read_msg_process (nxt_port_socket.c:1271)
  ==166470==    by 0x4055B3: nxt_port_read_handler (nxt_port_socket.c:778)
  ==166470==  Address 0x1ffefffc7f is on thread 1's stack
  ==166470==  in frame #3, created by nxt_unit_init (nxt_unit.c:428)
  ==166470==  Uninitialised value was created by a stack allocation
  ==166470==    at 0x4FEABFE: nxt_unit_init (nxt_unit.c:436)

  ==166690== Syscall param sendmsg(msg.msg_iov[0]) points to uninitialised byte(s)
  ==166690==    at 0x4AE6514: sendmsg (sendmsg.c:28)
  ==166690==    by 0x42D871: nxt_sendmsg (nxt_socket_msg.c:32)
  ==166690==    by 0x4FE6695: nxt_unit_sendmsg (nxt_unit.c:6009)
  ==166690==    by 0x4FE69C8: nxt_unit_port_send (nxt_unit.c:5939)
  ==166690==    by 0x4FE8C77: nxt_unit_request_done (nxt_unit.c:3309)
  ==166690==    by 0x4FEE13B: nxt_php_execute (nxt_php_sapi.c:1257)
  ==166690==    by 0x4FEE2F1: nxt_php_dynamic_request (nxt_php_sapi.c:1128)
  ==166690==    by 0x4FEE79E: nxt_php_request_handler (nxt_php_sapi.c:1023)
  ==166690==    by 0x4FE92AD: nxt_unit_process_ready_req (nxt_unit.c:4846)
  ==166690==    by 0x4FED1B4: nxt_unit_run_once_impl (nxt_unit.c:4605)
  ==166690==    by 0x4FED3AE: nxt_unit_run (nxt_unit.c:4548)
  ==166690==    by 0x4FEEC2A: nxt_php_start (nxt_php_sapi.c:514)
  ==166690==  Address 0x1ffeffea5f is on thread 1's stack
  ==166690==  in frame #3, created by nxt_unit_port_send (nxt_unit.c:5907)
  ==166690==  Uninitialised value was created by a stack allocation
  ==166690==    at 0x4FE8C05: nxt_unit_request_done (nxt_unit.c:3255)

These were due to the nxt_port_msg_t msg struct in nxt_unit_ready() and
nxt_unit_request_done() not being fully initialised.

Whether or not this is an actual problem an obviously correct thing to
do is to fully empty-initialise the structure and then we don't need to
explicitly set any members to 0 afterwards providing a nice cleanup as
well.

Link: <https://en.cppreference.com/w/c/language/initialization#Empty_initialization>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
ac000 added a commit that referenced this pull request Apr 9, 2025
valgrind(1) was producing the following alert

  ==166470== Syscall param sendmsg(msg.msg_control) points to uninitialised byte(s)
  ==166470==    at 0x4AE6514: sendmsg (sendmsg.c:28)
  ==166470==    by 0x42D86C: nxt_sendmsg (nxt_socket_msg.c:32)
  ==166470==    by 0x4FE6695: nxt_unit_sendmsg (nxt_unit.c:6013)
  ==166470==    by 0x4FEB6E2: nxt_unit_ready (nxt_unit.c:963)
  ==166470==    by 0x4FEB6E2: nxt_unit_init (nxt_unit.c:557)
  ==166470==    by 0x4FEEC56: nxt_php_start (nxt_php_sapi.c:507)
  ==166470==    by 0x426BA0: nxt_app_setup (nxt_application.c:1029)
  ==166470==    by 0x403153: nxt_process_do_start (nxt_process.c:718)
  ==166470==    by 0x4042A3: nxt_process_whoami_ok (nxt_process.c:846)
  ==166470==    by 0x407A28: nxt_port_rpc_handler (nxt_port_rpc.c:347)
  ==166470==    by 0x407E42: nxt_port_handler (nxt_port.c:184)
  ==166470==    by 0x40501B: nxt_port_read_msg_process (nxt_port_socket.c:1271)
  ==166470==    by 0x4055B3: nxt_port_read_handler (nxt_port_socket.c:778)
  ==166470==  Address 0x1ffefffc9c is on thread 1's stack
  ==166470==  in frame #3, created by nxt_unit_init (nxt_unit.c:428)
  ==166470==  Uninitialised value was created by a stack allocation
  ==166470==    at 0x4FEABFE: nxt_unit_init (nxt_unit.c:436)

This was due to the nxt_send_oob_t oob structure not being fully
initialised.

Given the name and intention of this function lets *fully*
empty-initialise this structure.

Link: <https://en.cppreference.com/w/c/language/initialization#Empty_initialization>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
ac000 added a commit that referenced this pull request Apr 9, 2025
valgrind(1) was producing the following alert

  ==166470== Syscall param sendmsg(msg.msg_control) points to uninitialised byte(s)
  ==166470==    at 0x4AE6514: sendmsg (sendmsg.c:28)
  ==166470==    by 0x42D86C: nxt_sendmsg (nxt_socket_msg.c:32)
  ==166470==    by 0x4FE6695: nxt_unit_sendmsg (nxt_unit.c:6013)
  ==166470==    by 0x4FEB6E2: nxt_unit_ready (nxt_unit.c:963)
  ==166470==    by 0x4FEB6E2: nxt_unit_init (nxt_unit.c:557)
  ==166470==    by 0x4FEEC56: nxt_php_start (nxt_php_sapi.c:507)
  ==166470==    by 0x426BA0: nxt_app_setup (nxt_application.c:1029)
  ==166470==    by 0x403153: nxt_process_do_start (nxt_process.c:718)
  ==166470==    by 0x4042A3: nxt_process_whoami_ok (nxt_process.c:846)
  ==166470==    by 0x407A28: nxt_port_rpc_handler (nxt_port_rpc.c:347)
  ==166470==    by 0x407E42: nxt_port_handler (nxt_port.c:184)
  ==166470==    by 0x40501B: nxt_port_read_msg_process (nxt_port_socket.c:1271)
  ==166470==    by 0x4055B3: nxt_port_read_handler (nxt_port_socket.c:778)
  ==166470==  Address 0x1ffefffc9c is on thread 1's stack
  ==166470==  in frame #3, created by nxt_unit_init (nxt_unit.c:428)
  ==166470==  Uninitialised value was created by a stack allocation
  ==166470==    at 0x4FEABFE: nxt_unit_init (nxt_unit.c:436)

This was due to the nxt_send_oob_t oob structure not being fully
initialised.

Given the name and intention of this function lets *fully*
empty-initialise this structure.

Link: <https://en.cppreference.com/w/c/language/initialization#Empty_initialization>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
ac000 added a commit that referenced this pull request Apr 11, 2025
valgrind(1) was producing the following alerts

  ==166470== Syscall param sendmsg(msg.msg_iov[0]) points to uninitialised byte(s)
  ==166470==    at 0x4AE6514: sendmsg (sendmsg.c:28)
  ==166470==    by 0x42D86C: nxt_sendmsg (nxt_socket_msg.c:32)
  ==166470==    by 0x4FE6695: nxt_unit_sendmsg (nxt_unit.c:6013)
  ==166470==    by 0x4FEB6E2: nxt_unit_ready (nxt_unit.c:963)
  ==166470==    by 0x4FEB6E2: nxt_unit_init (nxt_unit.c:557)
  ==166470==    by 0x4FEEC56: nxt_php_start (nxt_php_sapi.c:507)
  ==166470==    by 0x426BA0: nxt_app_setup (nxt_application.c:1029)
  ==166470==    by 0x403153: nxt_process_do_start (nxt_process.c:718)
  ==166470==    by 0x4042A3: nxt_process_whoami_ok (nxt_process.c:846)
  ==166470==    by 0x407A28: nxt_port_rpc_handler (nxt_port_rpc.c:347)
  ==166470==    by 0x407E42: nxt_port_handler (nxt_port.c:184)
  ==166470==    by 0x40501B: nxt_port_read_msg_process (nxt_port_socket.c:1271)
  ==166470==    by 0x4055B3: nxt_port_read_handler (nxt_port_socket.c:778)
  ==166470==  Address 0x1ffefffc7f is on thread 1's stack
  ==166470==  in frame #3, created by nxt_unit_init (nxt_unit.c:428)
  ==166470==  Uninitialised value was created by a stack allocation
  ==166470==    at 0x4FEABFE: nxt_unit_init (nxt_unit.c:436)

  ==166690== Syscall param sendmsg(msg.msg_iov[0]) points to uninitialised byte(s)
  ==166690==    at 0x4AE6514: sendmsg (sendmsg.c:28)
  ==166690==    by 0x42D871: nxt_sendmsg (nxt_socket_msg.c:32)
  ==166690==    by 0x4FE6695: nxt_unit_sendmsg (nxt_unit.c:6009)
  ==166690==    by 0x4FE69C8: nxt_unit_port_send (nxt_unit.c:5939)
  ==166690==    by 0x4FE8C77: nxt_unit_request_done (nxt_unit.c:3309)
  ==166690==    by 0x4FEE13B: nxt_php_execute (nxt_php_sapi.c:1257)
  ==166690==    by 0x4FEE2F1: nxt_php_dynamic_request (nxt_php_sapi.c:1128)
  ==166690==    by 0x4FEE79E: nxt_php_request_handler (nxt_php_sapi.c:1023)
  ==166690==    by 0x4FE92AD: nxt_unit_process_ready_req (nxt_unit.c:4846)
  ==166690==    by 0x4FED1B4: nxt_unit_run_once_impl (nxt_unit.c:4605)
  ==166690==    by 0x4FED3AE: nxt_unit_run (nxt_unit.c:4548)
  ==166690==    by 0x4FEEC2A: nxt_php_start (nxt_php_sapi.c:514)
  ==166690==  Address 0x1ffeffea5f is on thread 1's stack
  ==166690==  in frame #3, created by nxt_unit_port_send (nxt_unit.c:5907)
  ==166690==  Uninitialised value was created by a stack allocation
  ==166690==    at 0x4FE8C05: nxt_unit_request_done (nxt_unit.c:3255)

These were due to the nxt_port_msg_t msg struct in nxt_unit_ready() and
nxt_unit_request_done() not being fully initialised.

Whether or not this is an actual problem an obviously correct thing to
do is to fully empty-initialise the structure and then we don't need to
explicitly set any members to 0 afterwards providing a nice cleanup as
well.

Link: <https://en.cppreference.com/w/c/language/initialization#Empty_initialization>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
ac000 added a commit that referenced this pull request Apr 11, 2025
valgrind(1) was producing the following alert

  ==166470== Syscall param sendmsg(msg.msg_control) points to uninitialised byte(s)
  ==166470==    at 0x4AE6514: sendmsg (sendmsg.c:28)
  ==166470==    by 0x42D86C: nxt_sendmsg (nxt_socket_msg.c:32)
  ==166470==    by 0x4FE6695: nxt_unit_sendmsg (nxt_unit.c:6013)
  ==166470==    by 0x4FEB6E2: nxt_unit_ready (nxt_unit.c:963)
  ==166470==    by 0x4FEB6E2: nxt_unit_init (nxt_unit.c:557)
  ==166470==    by 0x4FEEC56: nxt_php_start (nxt_php_sapi.c:507)
  ==166470==    by 0x426BA0: nxt_app_setup (nxt_application.c:1029)
  ==166470==    by 0x403153: nxt_process_do_start (nxt_process.c:718)
  ==166470==    by 0x4042A3: nxt_process_whoami_ok (nxt_process.c:846)
  ==166470==    by 0x407A28: nxt_port_rpc_handler (nxt_port_rpc.c:347)
  ==166470==    by 0x407E42: nxt_port_handler (nxt_port.c:184)
  ==166470==    by 0x40501B: nxt_port_read_msg_process (nxt_port_socket.c:1271)
  ==166470==    by 0x4055B3: nxt_port_read_handler (nxt_port_socket.c:778)
  ==166470==  Address 0x1ffefffc9c is on thread 1's stack
  ==166470==  in frame #3, created by nxt_unit_init (nxt_unit.c:428)
  ==166470==  Uninitialised value was created by a stack allocation
  ==166470==    at 0x4FEABFE: nxt_unit_init (nxt_unit.c:436)

This was due to the nxt_send_oob_t oob structure not being fully
initialised.

Given the name and intention of this function lets *fully*
empty-initialise this structure.

Link: <https://en.cppreference.com/w/c/language/initialization#Empty_initialization>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
ac000 added a commit that referenced this pull request Aug 22, 2025
When doing some testing I was noticing when using brotli & zstd
compression on application responses we were regularly (but not always)
getting segfaults with

  "corrupted double-linked list"

being logged from malloc(3) when we were freeing memory via
nxt_mp_destroy() when doing nxt_router_http_request_release().

E.g.

  nginx#5  0x00007f6eeb4f11f5 in malloc_printerr (
      str=str@entry=0x7f6eeb625178 "corrupted double-linked list")
      at malloc.c:5829
  nginx#6  0x00007f6eeb4f1d0c in unlink_chunk (p=<optimized out>, av=0x7f6edc000030)
      at malloc.c:1619
  nginx#7  0x00007f6eeb4f1f78 in _int_free_create_chunk (av=av@entry=0x7f6edc000030,
      p=p@entry=0x7f6edc008ea0, size=size@entry=4192, nextchunk=<optimized out>,
      nextsize=75520) at malloc.c:4763
  nginx#8  0x00007f6eeb4f352e in _int_free_merge_chunk (av=av@entry=0x7f6edc000030,
      p=0x7f6edc008ea0, size=4192) at malloc.c:4742
  nginx#9  0x00007f6eeb4f36e4 in _int_free_chunk (av=0x7f6edc000030,
      p=<optimized out>, size=<optimized out>, have_lock=<optimized out>,
      have_lock@entry=0) at malloc.c:4667
  nginx#10 0x00007f6eeb4f6512 in _int_free (av=<optimized out>, p=<optimized out>,
      have_lock=0) at malloc.c:4699
  nginx#11 __GI___libc_free (mem=<optimized out>) at malloc.c:3476
  nginx#12 0x000000000040d66a in nxt_mp_destroy (mp=0x7f6edc003790)
      at src/nxt_mp.c:342
  nginx#13 0x000000000040d5a4 in nxt_mp_release (mp=0x7f6edc003790)
      at src/nxt_mp.c:303
  nginx#14 0x000000000042f9de in nxt_router_http_request_release (task=0x24cb8c10,
    obj=0x7f6edc003990, data=0x0) at src/nxt_router.c:5799

Interestingly gzip compression never seemed to trigger this...

Also when doing brotli compression for example, I could prevent this
from happening by simply commenting out

        BrotliEncoderDestroyInstance(brotli);

in src/nxt_brotli.c::nxt_brotli_compress()

Running under libasan showed the following

  ==281177==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7b94031e90f0 at pc 0x000000422b37 bp 0x7b640027c820 sp 0x7b640027c818
  READ of size 4 at 0x7b94031e90f0 thread T2
      #0 0x000000422b36 in nxt_buf_parent_completion src/nxt_buf.c:229
      #1 0x000000422d5e in nxt_buf_ts_completion src/nxt_buf.c:294
      #2 0x000000428fa0 in nxt_event_engine_start src/nxt_event_engine.c:542
      #3 0x0000004423de in nxt_router_thread_start src/nxt_router.c:3727
      nginx#4 0x00000042497b in nxt_thread_trampoline src/nxt_thread.c:126
      nginx#5 0x7f6404828ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
      nginx#6 0x7f640446f153 in start_thread (/lib64/libc.so.6+0x71153) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)
      nginx#7 0x7f64044f1cab in __clone3 (/lib64/libc.so.6+0xf3cab) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)

  0x7b94031e90f0 is located 8 bytes after 24-byte region [0x7b94031e90d0,0x7b94031e90e8)
  allocated by thread T2 here:
      #0 0x7f64048e6f2b in malloc (/lib64/libasan.so.8+0xe6f2b) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
      #1 0x000000401b10 in nxt_malloc src/nxt_malloc.c:35
      #2 0x000000401bd8 in nxt_zalloc src/nxt_malloc.c:54
      #3 0x000000410035 in nxt_port_incoming_port_mmap src/nxt_port_memory.c:247
      nginx#4 0x0000004162fa in nxt_port_mmap_handler src/nxt_port.c:366
      nginx#5 0x000000415000 in nxt_port_handler src/nxt_port.c:184
      nginx#6 0x00000040a761 in nxt_port_read_msg_process src/nxt_port_socket.c:1271
      nginx#7 0x00000040d596 in nxt_port_queue_read_handler src/nxt_port_socket.c:997
      nginx#8 0x000000428fa0 in nxt_event_engine_start src/nxt_event_engine.c:542
      nginx#9 0x0000004423de in nxt_router_thread_start src/nxt_router.c:3727
      nginx#10 0x00000042497b in nxt_thread_trampoline src/nxt_thread.c:126
      nginx#11 0x7f6404828ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)

  Thread T2 created by T0 here:
      #0 0x7f64048de492 in pthread_create (/lib64/libasan.so.8+0xde492) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
      #1 0x00000042468b in nxt_thread_create src/nxt_thread.c:85
      #2 0x00000044b799 in nxt_router_thread_create src/nxt_router.c:3575
      #3 0x00000044b799 in nxt_router_threads_create src/nxt_router.c:3543
      nginx#4 0x00000044b799 in nxt_router_conf_apply src/nxt_router.c:1271
      nginx#5 0x000000428fa0 in nxt_event_engine_start src/nxt_event_engine.c:542
      nginx#6 0x00000040140d in main src/nxt_main.c:35
      nginx#7 0x7f6404401574 in __libc_start_call_main (/lib64/libc.so.6+0x3574) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)
      nginx#8 0x7f6404401627 in __libc_start_main_alias_1 (/lib64/libc.so.6+0x3627) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)
      nginx#9 0x000000401264 in _start (/opt/unit/sbin/unitd+0x401264) (BuildId: c05bd11884a7315b24ec2abf762c4f283def6fea)

  SUMMARY: AddressSanitizer: heap-buffer-overflow src/nxt_buf.c:229 in nxt_buf_parent_completion
  Shadow bytes around the buggy address:
    0x7b94031e8e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e8e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e8f00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e8f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa 00 00
  =>0x7b94031e9080: 00 fa fa fa 00 00 00 05 fa fa 00 00 00 fa[fa]fa
    0x7b94031e9100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  Shadow byte legend (one shadow byte represents 8 application bytes):
    Addressable:           00
    Partially addressable: 01 02 03 04 05 06 07
    Heap left redzone:       fa
    Freed heap region:       fd
    Stack left redzone:      f1
    Stack mid redzone:       f2
    Stack right redzone:     f3
    Stack after return:      f5
    Stack use after scope:   f8
    Global redzone:          f9
    Global init order:       f6
    Poisoned by user:        f7
    Container overflow:      fc
    Array cookie:            ac
    Intra object redzone:    bb
    ASan internal:           fe
    Left alloca redzone:     ca
    Right alloca redzone:    cb
  ==281177==ABORTING

"SUMMARY: AddressSanitizer: heap-buffer-overflow src/nxt_buf.c:229 in
nxt_buf_parent_completion"

Gave some clue.

It seems that setting buf->parent on the last buffer triggers this.

If we don't set it on the last buffer, everything works fine and no
heap-overflow detected.

Everything seems to also work fine if we simply don't set it all. So
lets do that.

Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
ac000 added a commit that referenced this pull request Aug 25, 2025
When doing some testing I was noticing when using brotli & zstd
compression on application responses we were regularly (but not always)
getting segfaults with

  "corrupted double-linked list"

being logged from malloc(3) when we were freeing memory via
nxt_mp_destroy() when doing nxt_router_http_request_release().

E.g.

  nginx#5  0x00007f6eeb4f11f5 in malloc_printerr (
      str=str@entry=0x7f6eeb625178 "corrupted double-linked list")
      at malloc.c:5829
  nginx#6  0x00007f6eeb4f1d0c in unlink_chunk (p=<optimized out>, av=0x7f6edc000030)
      at malloc.c:1619
  nginx#7  0x00007f6eeb4f1f78 in _int_free_create_chunk (av=av@entry=0x7f6edc000030,
      p=p@entry=0x7f6edc008ea0, size=size@entry=4192, nextchunk=<optimized out>,
      nextsize=75520) at malloc.c:4763
  nginx#8  0x00007f6eeb4f352e in _int_free_merge_chunk (av=av@entry=0x7f6edc000030,
      p=0x7f6edc008ea0, size=4192) at malloc.c:4742
  nginx#9  0x00007f6eeb4f36e4 in _int_free_chunk (av=0x7f6edc000030,
      p=<optimized out>, size=<optimized out>, have_lock=<optimized out>,
      have_lock@entry=0) at malloc.c:4667
  nginx#10 0x00007f6eeb4f6512 in _int_free (av=<optimized out>, p=<optimized out>,
      have_lock=0) at malloc.c:4699
  nginx#11 __GI___libc_free (mem=<optimized out>) at malloc.c:3476
  nginx#12 0x000000000040d66a in nxt_mp_destroy (mp=0x7f6edc003790)
      at src/nxt_mp.c:342
  nginx#13 0x000000000040d5a4 in nxt_mp_release (mp=0x7f6edc003790)
      at src/nxt_mp.c:303
  nginx#14 0x000000000042f9de in nxt_router_http_request_release (task=0x24cb8c10,
    obj=0x7f6edc003990, data=0x0) at src/nxt_router.c:5799

Interestingly gzip compression never seemed to trigger this...

Also when doing brotli compression for example, I could prevent this
from happening by simply commenting out

        BrotliEncoderDestroyInstance(brotli);

in src/nxt_brotli.c::nxt_brotli_compress()

Running under libasan showed the following

  ==281177==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7b94031e90f0 at pc 0x000000422b37 bp 0x7b640027c820 sp 0x7b640027c818
  READ of size 4 at 0x7b94031e90f0 thread T2
      #0 0x000000422b36 in nxt_buf_parent_completion src/nxt_buf.c:229
      #1 0x000000422d5e in nxt_buf_ts_completion src/nxt_buf.c:294
      #2 0x000000428fa0 in nxt_event_engine_start src/nxt_event_engine.c:542
      #3 0x0000004423de in nxt_router_thread_start src/nxt_router.c:3727
      nginx#4 0x00000042497b in nxt_thread_trampoline src/nxt_thread.c:126
      nginx#5 0x7f6404828ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
      nginx#6 0x7f640446f153 in start_thread (/lib64/libc.so.6+0x71153) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)
      nginx#7 0x7f64044f1cab in __clone3 (/lib64/libc.so.6+0xf3cab) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)

  0x7b94031e90f0 is located 8 bytes after 24-byte region [0x7b94031e90d0,0x7b94031e90e8)
  allocated by thread T2 here:
      #0 0x7f64048e6f2b in malloc (/lib64/libasan.so.8+0xe6f2b) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
      #1 0x000000401b10 in nxt_malloc src/nxt_malloc.c:35
      #2 0x000000401bd8 in nxt_zalloc src/nxt_malloc.c:54
      #3 0x000000410035 in nxt_port_incoming_port_mmap src/nxt_port_memory.c:247
      nginx#4 0x0000004162fa in nxt_port_mmap_handler src/nxt_port.c:366
      nginx#5 0x000000415000 in nxt_port_handler src/nxt_port.c:184
      nginx#6 0x00000040a761 in nxt_port_read_msg_process src/nxt_port_socket.c:1271
      nginx#7 0x00000040d596 in nxt_port_queue_read_handler src/nxt_port_socket.c:997
      nginx#8 0x000000428fa0 in nxt_event_engine_start src/nxt_event_engine.c:542
      nginx#9 0x0000004423de in nxt_router_thread_start src/nxt_router.c:3727
      nginx#10 0x00000042497b in nxt_thread_trampoline src/nxt_thread.c:126
      nginx#11 0x7f6404828ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)

  Thread T2 created by T0 here:
      #0 0x7f64048de492 in pthread_create (/lib64/libasan.so.8+0xde492) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
      #1 0x00000042468b in nxt_thread_create src/nxt_thread.c:85
      #2 0x00000044b799 in nxt_router_thread_create src/nxt_router.c:3575
      #3 0x00000044b799 in nxt_router_threads_create src/nxt_router.c:3543
      nginx#4 0x00000044b799 in nxt_router_conf_apply src/nxt_router.c:1271
      nginx#5 0x000000428fa0 in nxt_event_engine_start src/nxt_event_engine.c:542
      nginx#6 0x00000040140d in main src/nxt_main.c:35
      nginx#7 0x7f6404401574 in __libc_start_call_main (/lib64/libc.so.6+0x3574) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)
      nginx#8 0x7f6404401627 in __libc_start_main_alias_1 (/lib64/libc.so.6+0x3627) (BuildId: 126a08bf502f4950b215dc773e52df8dcf50c393)
      nginx#9 0x000000401264 in _start (/opt/unit/sbin/unitd+0x401264) (BuildId: c05bd11884a7315b24ec2abf762c4f283def6fea)

  SUMMARY: AddressSanitizer: heap-buffer-overflow src/nxt_buf.c:229 in nxt_buf_parent_completion
  Shadow bytes around the buggy address:
    0x7b94031e8e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e8e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e8f00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e8f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa 00 00
  =>0x7b94031e9080: 00 fa fa fa 00 00 00 05 fa fa 00 00 00 fa[fa]fa
    0x7b94031e9100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    0x7b94031e9300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  Shadow byte legend (one shadow byte represents 8 application bytes):
    Addressable:           00
    Partially addressable: 01 02 03 04 05 06 07
    Heap left redzone:       fa
    Freed heap region:       fd
    Stack left redzone:      f1
    Stack mid redzone:       f2
    Stack right redzone:     f3
    Stack after return:      f5
    Stack use after scope:   f8
    Global redzone:          f9
    Global init order:       f6
    Poisoned by user:        f7
    Container overflow:      fc
    Array cookie:            ac
    Intra object redzone:    bb
    ASan internal:           fe
    Left alloca redzone:     ca
    Right alloca redzone:    cb
  ==281177==ABORTING

"SUMMARY: AddressSanitizer: heap-buffer-overflow src/nxt_buf.c:229 in
nxt_buf_parent_completion"

Gave some clue.

It seems that setting buf->parent on the last buffer triggers this.

If we don't set it on the last buffer, everything works fine and no
heap-overflow detected.

Everything seems to also work fine if we simply don't set it all. So
lets do that.

Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants