End-of-line characters prevent certain matches #78
Description
Zircolite seems to capture the \n end-of-line characters in the log fields it processes to perform its mapping from SIGMA rules. This can cause problems when mapping certain logs to Techniques.
For example, when I run the following command:
$ python3 zircolite.py --events audit.log --ruleset rules/rules_linux.json --auditd
I get the this (partial) output in the detected_events.json file:
{
"row_id": 2531,
"node": "n12-vm3",
"type": "CONFIG_CHANGE",
"timestamp": "2021-05-09 12:22:00",
"auid": "4294967295",
"ses": "4294967295",
"key": "susp_activity",
"host": "offline",
"OriginalLogfile": "audit.log-4RCGK4HT.json",
"op": "add_rule",
"res": "1\n",
"list": "4"
},
{
"row_id": 2535,
"node": "n12-vm3",
"type": "CONFIG_CHANGE",
"timestamp": "2021-05-09 12:22:00",
"auid": "4294967295",
"ses": "4294967295",
"key": "susp_activity",
"host": "offline",
"OriginalLogfile": "audit.log-4RCGK4HT.json",
"op": "add_rule",
"res": "1\n",
"list": "4"
},Here's a concrete example (extracted from my log file) of a log that should be triggered by Zircolite but isn't:
node=n12-vm3 type=SYSCALL msg=audit(1620555801.506:817): arch=c000003e syscall=59 success=yes exit=0 a0=5611f5511fa8 a1=5611f5511ed8 a2=5611f5511f78 a3=4040 items=2 ppid=1347 pid=1384 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="curl" exe="/usr/bin/curl" key="susp_activity"
While the following rule exists in the rules/rules_linux.json file:
{
"title": "Suspicious C2 Activities",
"id": "f7158a64-6204-4d6d-868a-6e6378b467e0",
"description": "Detects suspicious activities as declared by Florian Roth in its 'Best Practice Auditd Configuration'.\nThis includes the detection of the following commands; wget, curl, base64, nc, netcat, ncat, ssh, socat, wireshark, rawshark, rdesktop, nmap.\nThese commands match a few techniques from the tactics \"Command and Control\", including not exhaustively the following; Application Layer Protocol (T1071), Non-Application Layer Protocol (T1095), Data Encoding (T1132)\n",
"author": "Marie Euler",
"tags": [
"attack.command_and_control"
],
"falsepositives": [
"Admin or User activity"
],
"level": "medium",
"rule": [
"SELECT * FROM logs WHERE key LIKE 'susp\\_activity' ESCAPE '\\'"
],
"filename": "lnx_auditd_susp_c2_commands.yml"
}