Skip to content

Cloud Auth plugin & Google Pub/Sub destination #4651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Oct 6, 2023
Merged

Cloud Auth plugin & Google Pub/Sub destination #4651

merged 13 commits into from
Oct 6, 2023

Conversation

alltilla
Copy link
Collaborator

@alltilla alltilla commented Sep 28, 2023 •
edited
Loading

cloud-auth(): Added a new plugin for drivers, which implements different cloud related authentications.

Currently the only supported authentication is GCP's Service Account for the http() destination.

Example config:

http(
  cloud-auth(
    gcp(
      service-account(
        key("/path/to/service-account-key.json")
        audience("https://pubsub.googleapis.com/google.pubsub.v1.Publisher")
      )
    )
  )
);

Sending messages to Google Pub/Sub

The google-pubsub() destination feeds Google Pub/Sub via the HTTP REST API.

Example config:

google-pubsub(
  project("syslog-ng-project")
  topic("syslog-ng-topic")
  auth(
    service-account(
      key("/path/to/service-account-key.json")
    )
  )
);

See the Google Pub/Sub documentation to learn more about configuring a service account.

Signed-off-by: Attila Szakacs attila.szakacs@axoflow.com

@kira-syslogng
Copy link
Contributor

Build FAILURE

@alltilla alltilla force-pushed the pubsub branch 2 times, most recently from c8f38bd to 2799ff0 Compare September 28, 2023 14:34
furiel
furiel reviewed Sep 28, 2023
View reviewed changes
@kira-syslogng
Copy link
Contributor

Build FAILURE

@kira-syslogng
Copy link
Contributor

Build FAILURE

MrAnno
MrAnno reviewed Oct 2, 2023
View reviewed changes
Copy link
Collaborator

@MrAnno MrAnno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM so far 👍

@alltilla
cloud-auth: add base framework
86feed7 
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
cloud-auth: add jwt-cpp source as submodule
5420247 
https://github.com/Thalhammer/jwt-cpp

Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
scripts: skip jwt-cpp in style checking
31d3e0c 
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
cloud-auth: add gcp(service-account()) auth
71e765f 
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
scl: add google-pubsub() destination
f205276 
Exampel config:

google-pubsub(
  project("test-project-389014")
  topic("syslog-ng-test")
  auth(
    service-account(
      key("/path/to/test-project-389014-64d0bf69bec7.json")
      audience("https://pubsub.googleapis.com/google.pubsub.v1.Publisher")
    )
  )
);

Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
cloud-auth: log internal error if auth mode is not set
5e0d1ee 
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
cloud-auth: add token-validity-duration() to gcp(service-account())
511af88 
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
cloud-auth: allow setting multiple modes in gcp()
c12f11c 
It is useful to set default values for some options.
For example setting the service-account(audience()) in an SCL.

Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
pubsub: add default service-account(audience())
87617f5 
So the user does not have to set it manually.

Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla alltilla force-pushed the pubsub branch 2 times, most recently from 7cb41a6 to d9d7b25 Compare October 2, 2023 11:04
@alltilla
Copy link
Collaborator Author

alltilla commented Oct 2, 2023

I have fixed the review comments, modified a bit on the configurability and added news entries. This is ready for review again. @MrAnno @furiel Could you kindly take a look again? Thanks!

@alltilla
cloud-auth: add a deinit method
3ceb527 
Necessary for idempotency in case of a config revert.

Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
cloud-auth: add asserts to init()
c5711dc 
The lifecycle management of the cpp object must be done in init()
and deinit(), so we can be idempotent.

Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
cloud-auth: call the ssc's DISCONNECT()
af26e6c 
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
news: add entries for syslog-ng#4561
ba44191 
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
@alltilla
Copy link
Collaborator Author

alltilla commented Oct 2, 2023

Sry, pushed again.

@MrAnno MrAnno merged commit c7a28a8 into syslog-ng:master Oct 6, 2023
@eldarnash eldarnash mentioned this pull request Nov 25, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HofiOne HofiOne HofiOne left review comments

+2 more reviewers

@MrAnno MrAnno MrAnno approved these changes

@furiel furiel furiel approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@alltilla @kira-syslogng @MrAnno @furiel @HofiOne