[HAPROXY] use $SOURCEIP instead of $PROXIED_SRCIP #5305

HofiOne · 2025-03-28T16:21:13Z

This is a large refactor of the HAProxy support, and preparations for protocol auto detection. It also changes the HAProxy support to use the standard source/destination addresses in LogMessage, instead of a proxy specific values e.g. $SOURCEIP instead of $PROXIED_SRCIP.

Short summary of the patches:

small refactors and cleanups (first 5 patches)
simplify TLS compression setup code, previously this crossed a lot of layers, whereas it can be a lot simpler
LogTransportTLS is becoming an adapter, e.g. instead of going directly to file descriptors, go through another layer of the LogTransport interface. This prepares for protocol auto detection.
introduces LogTransportStack level aux data, making it possible to enrich messages from the any of the Transports
adds the $SOURCEPORT macro
LogTransportHAProxy is changed so it invokes a log_transport_stack_switch() instead of doing this internally
has a big refactor of TransportMapperInet, as its various settings and options were becoming too difficult to read. This patch also adds a large light testcase to cover all possible transport() cases
contains an alternative fix for 482 (although pretty similar)
and few smaller patches

Backport of 361 by @bazsi

Depend on #5303

kira-syslogng · 2025-03-28T17:25:06Z

Build FAILURE

kira-syslogng · 2025-04-04T11:34:15Z

Build FAILURE

HofiOne · 2025-04-07T08:52:28Z

@kira-syslogng test this please test=functions/source-drivers/file-source-driver/multiple-lines;

kira-syslogng · 2025-04-07T09:35:27Z

Build FAILURE

folti · 2025-04-12T20:25:46Z

@kira-syslogng retest this please

kira-syslogng · 2025-04-12T21:11:17Z

Build FAILURE

kira-syslogng · 2025-04-17T14:10:51Z

Build FAILURE

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

…ransportStack Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Previously TLS compression was enabled using an overly complicated mechanism crossing a number of layers (TransportMapperInet -> TransportFactoryTLS -> TLSSession -> SSL). This can be a lot simpler, which this patch implements. NOTE: compression will not work in most cases due to OpenSSL security levels and this patch adds a warning about it. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Instead of going to the fd directly, wrap the lower-level LogTransport instance into a BIO and use that. This implements proper stacking for LogTransportTLS. This adds the use of OpenSSL BIOs to wrap the lower level LogTransport instance. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

…rtStack level Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

…sages The "auto" protocol can be applied to both syslog() and network(), so it's not strictly RFC6587 related and it does not add too much information anyway. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Instead of using proxy protocol specific name value pairs, set the addresses in the message's saddr/daddr members. This should be a lot faster and a lot easier to use. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

…resses Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

…e_index Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

…e message Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

This reworks the various boolean members in TransportMapperInet that control which logproto/transport we apply to a specific connection. With these renames, it's much easier to follow what happens and why. NOTE: there's a followup bugfix that fixes the same bug as syslog-ng#482. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

"auto" has originally been planned to auto-detect TLS as well as framing format, but at this point it does not do TLS auto-detection. But this means that transport(auto) with tls() options set will start reading data without SSL, e.g. the encrypted stuff will make it into the messages received. This patch fixes that for both the syslog() and the network() driver. The only change is that delegate_tls_start_to_logproto is FALSE for the "auto" case. This will be changed once the TLS auto detection feature is also in. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

… transports Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

This was a one-off allocation, but it's better if it is freed. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Instead of just exercising the proxyprotocol try all valid transports, including the "auto" variants. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

kira-syslogng · 2025-04-23T12:53:23Z

Build FAILURE

HofiOne force-pushed the haproxy-use-sourceip-instead-of-proxied-srcip branch 6 times, most recently from 03f19e9 to 9f2db58 Compare March 28, 2025 16:41

HofiOne force-pushed the haproxy-use-sourceip-instead-of-proxied-srcip branch 2 times, most recently from d975302 to f27454d Compare April 4, 2025 10:52

HofiOne mentioned this pull request Apr 4, 2025

network(), syslog() sources: add $PEERIP and $PEERPORT macros #5291

Merged

HofiOne marked this pull request as ready for review April 11, 2025 10:07

HofiOne force-pushed the haproxy-use-sourceip-instead-of-proxied-srcip branch from f27454d to 32a900e Compare April 17, 2025 13:28

syslog-ng deleted a comment from github-actions bot Apr 22, 2025

bazsi added 11 commits April 23, 2025 14:10

light: allow the use of "auto" transport for syslog() source drivers

e5affed

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

logproto: rename prepare() methods to poll_prepare

5259267

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-stack: add poll_prepare() methods for LogTransport and LogT…

8310e71

…ransportStack Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-stack: add LogTransportStack level read/write/writev methods

5962341

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-adapter: add free_method

5180d10

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-aux-data: add log_transport_aux_data_move()

d616f3f

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-stack: make it possible to add aux data from the LogTranspo…

ce3a7e2

…rtStack level Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

logproto: call transport-stack level I/O methods

7676c39

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

bazsi added 12 commits April 23, 2025 14:10

templates: add $SOURCEPORT macro

1398e69

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-haproxy: use the LogTransportStack->aux to save src/dst add…

23bfdcd

…resses Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-haproxy: implement transport switch instead of changing bas…

9ec5c3d

…e_index Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport-haproxy: add more information to the proxy detection failur…

d0f24a8

…e message Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport/transport-factory-haproxy: use factory for creating HAProxy…

116ec31

… transports Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

transport: free LogTransportBIO BIO_meth instance at shutdown

c4a1f20

This was a one-off allocation, but it's better if it is freed. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

light: generalize test_pp_network and test_pp_syslog test cases

77d93c4

Instead of just exercising the proxyprotocol try all valid transports, including the "auto" variants. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

light: fix up files included in the dist

59c10b9

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

news: added news file

5b2e723

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com> Signed-off-by: Hofi <hofione@gmail.com>

HofiOne force-pushed the haproxy-use-sourceip-instead-of-proxied-srcip branch from 32a900e to 5b2e723 Compare April 23, 2025 12:11

kovgeri01 approved these changes Apr 23, 2025

View reviewed changes

HofiOne merged commit f5ca76e into syslog-ng:develop Apr 23, 2025
29 of 30 checks passed

HofiOne deleted the haproxy-use-sourceip-instead-of-proxied-srcip branch April 23, 2025 13:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[HAPROXY] use $SOURCEIP instead of $PROXIED_SRCIP #5305

[HAPROXY] use $SOURCEIP instead of $PROXIED_SRCIP #5305

Uh oh!

HofiOne commented Mar 28, 2025 •

edited

Loading

Uh oh!

kira-syslogng commented Mar 28, 2025

Uh oh!

kira-syslogng commented Apr 4, 2025

Uh oh!

HofiOne commented Apr 7, 2025

Uh oh!

kira-syslogng commented Apr 7, 2025

Uh oh!

folti commented Apr 12, 2025

Uh oh!

kira-syslogng commented Apr 12, 2025

Uh oh!

kira-syslogng commented Apr 17, 2025

Uh oh!

kira-syslogng commented Apr 23, 2025

Uh oh!

Uh oh!

Uh oh!

[HAPROXY] use $SOURCEIP instead of $PROXIED_SRCIP #5305

[HAPROXY] use $SOURCEIP instead of $PROXIED_SRCIP #5305

Uh oh!

Conversation

HofiOne commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kira-syslogng commented Mar 28, 2025

Uh oh!

kira-syslogng commented Apr 4, 2025

Uh oh!

HofiOne commented Apr 7, 2025

Uh oh!

kira-syslogng commented Apr 7, 2025

Uh oh!

folti commented Apr 12, 2025

Uh oh!

kira-syslogng commented Apr 12, 2025

Uh oh!

kira-syslogng commented Apr 17, 2025

Uh oh!

kira-syslogng commented Apr 23, 2025

Uh oh!

Uh oh!

Uh oh!

HofiOne commented Mar 28, 2025 •

edited

Loading