Skip to content

Support for importing AWS ELB access logs analytics #131

@guilhermp

Description

@guilhermp

Hi,
I've been trying to parse AWS Elastic Load Balancer logs but without success so far. Does anyone know of a working regex to do this, or can suggest me a good document Piwik's log-format-regex?

The log-format-regex I've been trying to use is as follows:
--log-format-regex='.* ((?P<date>.*?) (?P<elb>.*?) (?P<ip>.*?) (?P<port>.*?) (?P<ip_r>.*?) (?P<port_r>.*?) (?P<proc_time>.*?) (?P<backend_proc_time>.*?) (?P<response_proc_time>.*?) (?P<status>.*?) (?P<backend_status>.*?) (?P<rd_bytes>\S+) (?P<length>\S+) "(?P<user_agent>.*?)" (?P<ssl_cipher>\S+) (?P<ssl_protocol>\S+)).*'

And here's a log sample I found on AWS documentation:

2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1" "curl/7.38.0" DHE-RSA-AES128-SHA TLSv1.2

I was able to parse the same logs on AWStats after converting timestamps with sed so that they look like:
2015-05-13 23:39:43 my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1" "curl/7.38.0" DHE-RSA-AES128-SHA TLSv1.2

AWStats Log Format:
LogFormat="%time2 %elb %host %host_port %host_r %host_r_port %request_processing_time %backend_processing_time %response_processing_time %code %other %received_bytes %bytesd %methodurl %uaquot %extra1 %extra2"

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions