Fluentbit installation and configuration role.
None.
Available variables are listed below along with default values (see defaults\main.yaml)
Default configuration of this role keeps most of default configuration file created by fluent-bit package installation. It only removes default INPUT (cpu monitoring) and OUTPUT (stdout) sections.
You can find more information about each option in the Fluentbit documentation.
- fluentbit_service_flush_seconds: Flush time in seconds [default: 5]
- fluentbit_service_enable_metrics: Enable metrics HTTP server [default: false]
- fluentbit_service_log_level: Set logging verbosity level [default: info]
For configuring INPUT, FILTER and OUTPUT section within fluentbit configuration the following variables are used
- fluentbit_inputs: Fluentbit INPUT configuration section [default: ""].
- fluentbit_filters: Fluentbit FILTER configuration section [default: ""].
- fluentbit_outputs: Fluentbit OUTPUT configuration section [default: ""].
Using that variables the config file content can be added using using yaml | operator
fluentbit_inputs: |
[INPUT]
name tail
tag auth
path /var/log/auth.log
path_key log_file
DB /run/fluent-bit-auth.state
Parser syslog-rfc3164
[INPUT]
name tail
tag syslog
path /var/log/syslog
path_key log_file
DB /run/fluent-bit-syslog.state
Parser syslog-rfc3164Default package fluent-bit installation comes with a predefined set of parsers included as part of parsers.conf file. Additional configuration files containing parsers can be added to fluent-bit configuration. This role uses custom_parser.conf file for keeping additional custom parsers.
Content of this custom parser file can be configured using the variable fluentbit_custom_parsers
fluentbit_custom_parsers: |
[PARSER]
Name syslog-rfc3164-nopri
Format regex
Regex /^(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
Time_Key time
Time_Format %b %d %H:%M:%S
Time_Keep FalseLua Filter allows you to modify the incoming records using custom Lua Scripts. See fluentbit documentation
Lua filter can be included in fluentbit_filters variable
fluentbit_filters: |
[FILTER]
Name lua
Match host.*
script /etc/fluent-bit/adjust_ts.lua
call local_timestamp_to_UTCLua scripts used by the filters need to be configured using fluentbit_lua_scripts role variable. This variable is a list of dictionaries with 2 fields:
name: name of the lua scriptcontent: script content)
Script content can be populated from a file using ansible lookup plugin.
fluentbit_lua_scripts:
- name: adjust_ts.lua
content: "{{ lookup('template','templates/adjust_ts.lua') }}"
and templates/adjust_ts.lua file content:
function local_timestamp_to_UTC(tag, timestamp, record)
local utcdate = os.date("!*t", ts)
local localdate = os.date("*t", ts)
localdate.isdst = false -- this is the trick
utc_time_diff = os.difftime(os.time(localdate), os.time(utcdate))
return 1, timestamp - utc_time_diff, record
endNOTE: all lua scripts will be copied to
/etc/fluent-bitdirectory
None
- name: Fluent-bit-installation
hosts: all
become: true
gather_facts: true
roles:
- role: ricsanfre.fluentbit
fluentbit_service_enable_metrics: true
# lua scripts
fluentbit_lua_scripts:
- name: adjust_ts.lua
content: "{{ lookup('template','templates/adjust_ts.lua') }}"
# inputs
fluentbit_inputs: |
[INPUT]
Name tail
Tag host.*
DB /run/fluentbit-state.db
Path /var/log/auth.log,/var/log/syslog
Parser syslog-rfc3164-nopri
# filters
fluentbit_filters: |
[FILTER]
Name lua
Match host.*
script /etc/fluent-bit/adjust_ts.lua
call local_timestamp_to_UTC
# outputs
fluentbit_outputs: |
[OUTPUT]
Name stdout
Match *
# Parsers
fluentbit_custom_parsers: |
[PARSER]
Name syslog-rfc3164-nopri
Format regex
Regex /^(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
Time_Key time
Time_Format %b %d %H:%M:%S
Time_Keep False