Skip to content

Multiple requests may archive the same range request at the same time #18088

@tsteur

Description

@tsteur

We noticed a while ago that multiple requests may launch archiving for the same period.

To reproduce this:

  • Disable browser archiving & Disable browser archiving for a segment
    • eg set [General]enable_browser_archiving_triggering = 0 enable_general_settings_admin = 0 browser_archiving_disabled_enforce = 1
  • Have a site with say 30 active goal
  • Have generally data for a few days
  • Trigger archiving using ./console core:archive
  • Then go the Goals Overview page and request a date range eg 2021-09-02,2021,09,29

You might then notice that all does goals trigger the archiving. Eg you could execute a query like

select count(*) from archive_numeric_2021_09 where ts_archived > DATE_SUB(NOW(), INTERVAL 5 MINUTE) and ts_archived < NOW() and name like 'done%'; -- number of unique added archives in the last few minutes which may be pretty much roughly like the number of goals
select count(*) from archive_numeric_2021_09 where ts_archived > DATE_SUB(NOW(), INTERVAL 5 MINUTE) and ts_archived < NOW() ;// how many archive entries were added overall

-- same for blob
select count(*) from archive_blob_2021_09 where ts_archived > DATE_SUB(NOW(), INTERVAL 5 MINUTE) and ts_archived < NOW() ;

I'm meaning that each of these requests like in the screenshot seem to be launching the archiving (eg index.php?forceView=1&viewDataTable=sparklines&module=Goals&action=get&idGoal=7&allow_multiple=0&only_summary=1&idSite=12&period=week&date=today&segment=&showtitle=1&random=1482):

image

As I'm seeing this recently more often I think this was a regression somewhere between 4.3 and 4.4. I quickly looked through the archiving related changes in 4.3.0...4.4.1 and couldn't find anything so far that would explain this. This one peaked my interest but pretty sure it cannot be what caused it: https://github.com/matomo-org/matomo/pull/17657/files

Possibly the regression was in 4.2.0 because of https://github.com/matomo-org/matomo/pull/17379/files maybe

I wonder if this one maybe previously prevented that each request launches the archiving at the same time? Once we can reproduce the issue, we could maybe see if it works with that PR and wouldn't launch multiple archives.

Let me know if there are any issues reproducing this issue.

refs #17774

The goal is that the archiving is triggered only once for all these requests. Meaning below query should return 1:

select count(*) from archive_numeric_2021_09 where ts_archived > DATE_SUB(NOW(), INTERVAL 5 MINUTE) and ts_archived < NOW() and name like 'done%'; -- number of unique added archives in the last few minutes which may be pretty much roughly like the number of goals

Metadata

Metadata

Assignees

Labels

BugFor errors / faults / flaws / inconsistencies etc.RegressionIndicates a feature used to work in a certain way but it no longer does even though it should.not-in-changelogFor issues or pull requests that should not be included in our release changelog on matomo.org.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions