Invalid log lines with common_complete log format #197
Description
Hello, I'm trying to log entries from several virtual hosts, such as a.tile.openstreetmap, b.tile.openstreetmap, all stored in the same nginx log file. I've tried to follow the official documentation for this, adding a new the log format on nginx:
log_format vhosts '$host $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"';
with the following vhost configuration:
server {
listen 80;
listen [::]:80;
server_name openstreetmap.c3sl.ufpr.br *.tile.openstreetmap.c3sl.ufpr.br;
server_tokens off;
access_log /var/log/nginx/openstreetmap_access.log vhosts;
error_log /var/log/nginx/openstreetmap_error.log;
...
}
After setting things up like above, I can see log entries in openstreetmap_access.log like these:
a.tile.openstreetmap.c3sl.ufpr.br 10.254.221.79 - - [04/Dec/2017:18:37:32 -0200] "GET /osm/16/24767/36466.png HTTP/1.1" 200 20932 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"
c.tile.openstreetmap.c3sl.ufpr.br 10.254.221.79 - - [04/Dec/2017:18:37:32 -0200] "GET /osm/16/24767/36465.png HTTP/1.1" 200 20922 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"
c.tile.openstreetmap.c3sl.ufpr.br 10.254.221.79 - - [04/Dec/2017:18:37:32 -0200] "GET /osm/15/12384/18233.png HTTP/1.1" 200 24948 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"
b.tile.openstreetmap.c3sl.ufpr.br 10.254.221.79 - - [04/Dec/2017:18:37:32 -0200] "GET /osm/16/24767/36467.png HTTP/1.1" 200 19117 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"
c.tile.openstreetmap.c3sl.ufpr.br 10.254.221.79 - - [04/Dec/2017:18:37:32 -0200] "GET /osm/16/24766/36466.png HTTP/1.1" 200 14544 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"
Then I try to run the following command:
python import_logs.py --url piwik.c3sl.ufpr.br --idsite=7 --log-format-name=common_complete --token-auth=xxnot-this-timexx --enable-http-errors --enable-static /var/log/nginx/openstreetmap_access.log
and I get the following output:
Logs import summary
-------------------
221 requests imported successfully
0 requests were downloads
221 requests ignored:
0 HTTP errors
0 HTTP redirects
221 invalid log lines
0 filtered log lines
0 requests did not match any known site
0 requests did not match any --hostname
0 requests done by bots, search engines...
0 requests to static resources (css, js, images, ico, ttf...)
0 requests to file downloads did not match any --download-extensions
Website import summary
----------------------
221 requests imported to 1 sites
1 sites already existed
0 sites were created:
0 distinct hostnames did not match any existing site:
So, all lines were considered invalid. Why?
Environment:
- Piwik 3.2.0
- piwik-log-analytics: master (05a5154)
Thank you very much.