Unable to use ElasticSearch with Selfsigned Certs #4681
Description
I'm using logging-operator-4.4.1
Using below given SyslogNGOutput in logging-operator
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGOutput
metadata:
name: syslog-output
namespace: default
spec:
elasticsearch:
url: "https://elasticsearch.elastic.svc.cluster.local:9200/_bulk"
index: "expertflow"
type: ""
user: "elastic"
# tls.peer-verify: "no"
# tls.ssl-version: "TLSv1_2"
password:
valueFrom:
secretKeyRef:
name: elastic
key: passwordand this is my SyslogNGFlow
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGFlow
metadata:
name: flow-all
namespace: default
spec:
match:
regexp:
value: json.kubernetes.labels.app.kubernetes.io/instance
pattern: "*"
type: glob
localOutputRefs:
- syslog-output
Getting below given errors in syslogng pods
[2023-10-22T12:52:37.660629] Setting current version as config version; version='4.4'
[2023-10-22T12:52:37.680824] Error resolving reference; content='destination', name='output_default_syslog-output', location='/etc/syslog-ng/config/syslog-ng.conf:26:5'
[2023-10-22T12:52:37.680892] Error initializing new configuration, reverting to old config;
[2023-10-22T12:52:37.681402] Configuration reload finished;
[2023-10-22T12:56:07.694248] Setting current version as config version; version='4.4'
[2023-10-22T12:56:07.761653] Accepting connections; addr='AF_INET(0.0.0.0:601)'
[2023-10-22T12:56:07.774742] Configuration reload request received, reloading configuration;
[2023-10-22T12:56:07.774857] Configuration reload finished;
[2023-10-22T12:56:08.457956] Syslog connection accepted; fd='13', client='AF_INET(10.42.0.53:38684)', local='AF_INET(0.0.0.0:601)'
[2023-10-22T12:56:08.747919] Syslog connection accepted; fd='22', client='AF_INET(10.42.0.53:38700)', local='AF_INET(0.0.0.0:601)'
[2023-10-22T13:01:00.257717] Input is valid utf8, but the log message is not tagged as such, this performs worse than enabling validate-utf8 flag on input; value=''
[2023-10-22T13:01:00.380611] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='0', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:01:00.384529] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='0', time_reopen='60', batch_size='1'
[2023-10-22T13:01:03.975765] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='1', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:01:03.975914] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='1', time_reopen='60', batch_size='1'
[2023-10-22T13:01:05.341636] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='2', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:01:05.342419] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='2', time_reopen='60', batch_size='1'
[2023-10-22T13:01:14.415720] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='3', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:01:14.415900] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='3', time_reopen='60', batch_size='1'
[2023-10-22T13:02:00.864644] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='0', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:02:00.864873] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='0', time_reopen='60', batch_size='1'
[2023-10-22T13:02:04.167510] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='1', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:02:04.167685] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='1', time_reopen='60', batch_size='1'
[2023-10-22T13:02:05.902513] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='2', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:02:05.902658] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='2', time_reopen='60', batch_size='1'
[2023-10-22T13:02:14.668856] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='3', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:02:14.669053] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='3', time_reopen='60', batch_size='1'
[2023-10-22T13:03:01.227607] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='0', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:03:01.227768] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='0', time_reopen='60', batch_size='1'
[2023-10-22T13:03:04.360103] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='1', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:03:04.360219] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='1', time_reopen='60', batch_size='1'
[2023-10-22T13:03:06.071977] curl: error sending HTTP request; url='https://elasticsearch.elastic.svc.cluster.local:9200/_bulk', error='SSL peer certificate or SSH remote key was not OK', worker_index='2', driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5'
[2023-10-22T13:03:06.072160] Server disconnected while preparing messages for sending, trying again; driver='output_default_syslog-output#0', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:29:5', worker_index='2', time_reopen='60', batch_size='1'
I have disabled these tags in syslogngoutput
# tls.peer-verify: "no"
# tls.ssl-version: "TLSv1_2"because neither array nor object style works.
tried with
tls
peer-verify: "no"
ssl-version: "TLSv1_2"also
tls
- peer-verify: "no"
- ssl-version: "TLSv1_2"Any suggestion.
I'm running Elasticsearch using Selfsigned Certs on the same cluster. and it work
$ curl -kL -u "elastic:Elastic123" "https://elasticsearch.elastic.svc.cluster.local:9200/_cat/indices"
green open .internal.alerts-security.alerts-default-000001 GEyZBB2XStyTCadBAeuzGg 1 1 0 0 496b 248b
green open .internal.alerts-stack.alerts-default-000001 wGBrnidQTeegjcSuruM7Aw 1 1 0 0 496b 248b
green open .internal.alerts-observability.slo.alerts-default-000001 IILJg1zUTdCWA6_Rfpx2cg 1 1 0 0 496b 248b
green open .internal.alerts-observability.uptime.alerts-default-000001 tly4DqOzR6WaIUdlqCCpxg 1 1 0 0 496b 248b
green open .internal.alerts-observability.apm.alerts-default-000001 B4MFc7tqRECGAljqW1fUBw 1 1 0 0 496b 248b
green open .internal.alerts-observability.logs.alerts-default-000001 hWk9OIW3SQqjIFePcpybaA 1 1 0 0 496b 248b
green open .kibana-observability-ai-assistant-conversations-000001 CHW-LezjSzyjVJxKM0OR-g 1 1 0 0 496b 248b
green open .internal.alerts-observability.metrics.alerts-default-000001 ontO9zExQnyE3CnhYTbuoA 1 1 0 0 496b 248b
[ root@curly:/ ]$ exit