BGP container crashes after BGP established with 500/1K IPv4 and IPv6 peers #14143
Description
Describe the bug
Establish 2k (1k IPv4, 1k IPv6) dynamic BGP sessions and wait for a minute. Observe bgpd crash. The crash is observed only when using SNMP module: -M snmp option in bgpd command line.
The log:
Feb 27 13:54:36.185873 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: Received signal 6 at 1677506076 (si_addr 0x12c00000038, PC 0x7fccf3f46ce1); aborting...
Feb 27 13:54:36.186112 qa-eth-vt03-1-4600ca1 INFO bgp#supervisord: bgpd *** buffer overflow detected ***: terminated
Feb 27 13:54:36.186314 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: zlog_signal+0xf5 7fccf42fc215 7ffdfd035230 /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0 (mapped at 0x7fccf4260000)
Feb 27 13:54:36.186580 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: PBKDF2_SHA256+0x4e1 7fccf4328851 7ffdfd035370 /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0 (mapped at 0x7fccf4260000)
Feb 27 13:54:36.186852 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: funlockfile+0x50 7fccf40f6140 7ffdfd0354c0 /lib/x86_64-linux-gnu/libpthread.so.0 (mapped at 0x7fccf40e3000)
Feb 27 13:54:36.187158 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: ---- signal ----
Feb 27 13:54:36.187185 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: gsignal+0x141 7fccf3f46ce1 7ffdfd035a70 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7fccf3f0e000)
Feb 27 13:54:36.187432 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: abort+0x123 7fccf3f30537 7ffdfd035b90 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7fccf3f0e000)
Feb 27 13:54:36.187724 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: __fsetlocking+0x288 7fccf3f89768 7ffdfd035cc0 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7fccf3f0e000)
Feb 27 13:54:36.187992 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: __fortify_fail+0x22 7fccf401a542 7ffdfd035e00 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7fccf3f0e000)
Feb 27 13:54:36.188351 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: __chk_fail+0x10 7fccf4018f20 7ffdfd035e20 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7fccf3f0e000)
Feb 27 13:54:36.188567 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: __fdelt_warn+0x17 7fccf401a497 7ffdfd035e30 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7fccf3f0e000)
Feb 27 13:54:36.188764 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: ? 7fccf3c9a475 7ffdfd035e40 /usr/lib/x86_64-linux-gnu/frr/libfrrsnmp.so.0 (mapped at 0x7fccf3c97000)
Feb 27 13:54:36.188919 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: ? 7fccf3c9a9e3 7ffdfd035f40 /usr/lib/x86_64-linux-gnu/frr/libfrrsnmp.so.0 (mapped at 0x7fccf3c97000)
Feb 27 13:54:36.189201 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: thread_call+0x7d 7fccf433a48d 7ffdfd035f50 /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0 (mapped at 0x7fccf4260000)
Feb 27 13:54:36.189461 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: frr_run+0xe8 7fccf42f44a8 7ffdfd035ff0 /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0 (mapped at 0x7fccf4260000)
Feb 27 13:54:36.189605 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: main+0x356 55d92d7c5286 7ffdfd036210 /usr/lib/frr/bgpd (mapped at 0x55d92d6e6000)
Feb 27 13:54:36.189928 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: __libc_start_main+0xea 7fccf3f31d0a 7ffdfd036270 /lib/x86_64-linux-gnu/libc.so.6 (mapped at 0x7fccf3f0e000)
Feb 27 13:54:36.190124 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: _start+0x2a 55d92d7c6f5a 7ffdfd036340 /usr/lib/frr/bgpd (mapped at 0x55d92d6e6000)
Feb 27 13:54:36.190160 qa-eth-vt03-1-4600ca1 CRIT bgp#BGP[56]: in thread agentx_timeout scheduled from ../lib/agentx.c:124 agentx_events_update()
Feb 27 13:54:36.190396 qa-eth-vt03-1-4600ca1 INFO bgp#supervisord: bgpd core_handler: showing active allocations in memory group libfrr
Feb 27 13:54:36.190557 qa-eth-vt03-1-4600ca1 INFO bgp#supervisord: bgpd core_handler: memstats: Buffer : 2 * 24
Feb 27 13:54:36.190557 qa-eth-vt03-1-4600ca1 INFO bgp#supervisord: bgpd core_handler: memstats: Host config : 5 *
- Did you check if this is a duplicate issue?
- Did you test it on the latest FRRouting/frr master branch?
To Reproduce
For example:
- Run bgpd with
-M snmpoption with SNMP agentx. - Establish 1k IPv4, 1k IPv6 dynamic BGP neighbors
Expected behavior
Expect it to work, no crash.
Screenshots
Versions
- OS Version: Debian GNU/Linux 11 (bullseye)
- Kernel: 5.10.0-18-2-amd64
- FRR Version: 8.2.2-sonic-0 (patched with https://github.com/sonic-net/sonic-buildimage/tree/202211/src/sonic-frr/patch)
Additional context
The issue happens on SONiC OS.