Skip to content

Faster & Reliable Tracking: piwik.php asynchronous tracking import, by replaying piwik.php access logs every N minutes #3632

@mattab

Description

@mattab

This is a performance improvement ticket.

  • It is possible to disable piwik.php from tracking (ie. not select/update/insert) from the DB. See the Faq to enable the maintenance mode: http://piwik.org/faq/how-to/#faq_111
  • When this is enabled, the webserver will return very quickly the transparent GIF. It will also log the piwik.php request in the apache server logs. It turns out this request contains (by design) the full set of information required by Tracking.
  • We should provide a new CRON script that will, replay the piwik.php server logs every 1 minute for example. We could make use of Piwik Bulk Tracking features to import ie. 500 lines at once from the log.

Summary:
Power users will be able to setup Piwik where the Mysql is not required for tracking.
A script will run that will import, every N minutes (for example N=1 or N=60) the Webserver access logs into Piwik.
This script will use our log analytics tool to import the piwik.php requests.
It won't be as "real time" as before since loading logs is asynchronous, but could be set every 1 minute for near real time.

This will make Piwik tracking decoupled from Mysql, more resilient, faster, easier to scale.

Metadata

Metadata

Assignees

Labels

EnhancementFor new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.c: PerformanceFor when we could improve the performance / speed of Matomo.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions