Skip to content

Create temporary table for segments only once instead of twice when archiving #14818

@tsteur

Description

@tsteur

Currently, when we archive, we initialize the PluginsArchiver twice. Once for core and once for the plugins here: https://github.com/matomo-org/matomo/blob/3.12.0-b2/core/ArchiveProcessor/Loader.php#L101-L117

In the PluginsArchiver's constructor we then first cleanup any possible temp table that already exists ( https://github.com/matomo-org/matomo/blob/3.12.0-b2/core/ArchiveProcessor/PluginsArchiver.php#L70-L71) and then later create the temp table as soon as a LogAggregator::generateQuery is called.

This means what we do is:

  • Start archiving core metrics

  • Cleanup temp table for segment if for some reason was not removed (eg a fatal error or memory issue etc and old MySQL version is used)

  • Create temp table for segment

  • archive core metrics

  • Finalise archive

  • Cleanup temp table for segment ===> This is the step we want to prevent but only if we actually go into the plugins archiver again to archive plugin metrics

  • Start archiving plugins metrics

  • Cleanup temp table for segment if for some reason was not removed (eg a fatal error or memory issue etc and old MySQL version is used) ===> This is the step we want to prevent

  • archive plugin metrics

  • Finalise archive

  • Cleanup temp table for segment

This will make segmenting again a bit faster as we avoid running the expensive segment query and avoid lots of inserts...

Metadata

Metadata

Assignees

Labels

c: PerformanceFor when we could improve the performance / speed of Matomo.not-in-changelogFor issues or pull requests that should not be included in our release changelog on matomo.org.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions