Skip to content

Add archiving diagnostic commands #21713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Dec 28, 2023
Merged

Add archiving diagnostic commands #21713

merged 8 commits into from
Dec 28, 2023

Conversation

bx80
Copy link
Contributor

@bx80 bx80 commented Dec 19, 2023 •
edited by caddoo
Loading

Description:

Fixes #21666

This PR adds five new diagnostic commands for archiving:

./console diagnostic:archiving-queue [--json]

Shows the current invalidation queue in a human readable format, optional output as json.

+--------------+------+--------+-------------------------+---------------------+---------------+---------+------------+--------+
| Invalidation | Site | Period | Date                    | Time Queued         | Waiting       | Started | Processing | Status |
+--------------+------+--------+-------------------------+---------------------+---------------+---------+------------+--------+
| 1            | 1    | Day    | 2010-03-06              | 2010-03-07 01:00:00 | 5 hours 0 min |         |            | Queued |
| 2            | 1    | Week   | 2010-03-01 - 2010-03-07 | 2010-03-07 01:00:00 | 5 hours 0 min |         |            | Queued |
| 3            | 1    | Month  | 2010-03                 | 2010-03-07 01:00:00 | 5 hours 0 min |         |            | Queued |
| 4            | 1    | Year   | 2010                    | 2010-03-07 01:00:00 | 5 hours 0 min |         |            | Queued |
+--------------+------+--------+-------------------------+---------------------+---------------+---------+------------+--------+

./console diagnostic:archiving-metrics [--json]

Shows a list of archiving related metrics, optional json output.

+--------------------------------------+---------------------+
| Metric                               | Value               |
+--------------------------------------+---------------------+
| Total Invalidation Count             | 4                   |
| In Progress Invalidation Count       | 0                   |
| Scheduled Invalidation Count         | 4                   |
| Earliest invalidation ts_started     |                     |
| Latest invalidation ts_started       |                     |
| Earliest invalidation ts_invalidated | 2010-03-07 01:00:00 |
| Latest invalidation ts_invalidated   | 2010-03-07 01:00:00 |
| Number of segment invalidations      | 0                   |
| Number of plugin invalidations       | 0                   |
| List of plugins being invalidated    |                     |
+--------------------------------------+---------------------+

./console diagnostic:archiving-instance-statistics [--json]

Show a list of statistics for the current server, option json output.

+----------------------+------------------------+
| Statistic Name       | Value                  |
+----------------------+------------------------+
| Site Count           | 3                      |
| Segment Count        | 1                      |
| Database Version     | mysql-version-redacted |
| Last full Month Hits | 8                      |
| Last 12 Month Hits   | 8                      |
+----------------------+------------------------+

./console diagnostic:archiving-config [--json]

Shows the value of all archiving related config settings, optional json output.

+----------+-------------------------------------------------------------+-------------------+
| Section  | Setting                                                     | Value             |
+----------+-------------------------------------------------------------+-------------------+
| database | enable_segment_first_table_join_prefix                      |                   |
| database | enable_first_table_join_prefix                              |                   |
| general  | browser_archiving_disabled_enforce                          | 0                 |
  ...
| general  | datatable_archiving_maximum_rows_subtable_events            | 500               |
| general  | datatable_archiving_maximum_rows_products                   | 10000             |
| general  | datatable_archiving_maximum_rows_standard                   | 500               |
+----------+-------------------------------------------------------------+-------------------+

./console diagnostic:archiving-status [--with-stats] [--email=EMAIL]

Combines the output of all of the above four commands, by default instance statistics are not included but can be added by using the --with-stats parameter. If the --email parameter is used to specific an email address then the output will be sent to that email, otherwise it will be displayed on the console.

image

Review

@bx80 bx80 added Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Easier debugging For issues that make troubleshooting issues for developers easier. labels Dec 19, 2023
@bx80 bx80 added this to the 5.0.1 milestone Dec 19, 2023
@bx80 bx80 self-assigned this Dec 19, 2023
michalkleiner
michalkleiner reviewed Dec 19, 2023
View reviewed changes
michalkleiner
michalkleiner reviewed Dec 19, 2023
View reviewed changes
michalkleiner
michalkleiner reviewed Dec 19, 2023
View reviewed changes
michalkleiner
michalkleiner reviewed Dec 19, 2023
View reviewed changes
sgiehl
sgiehl requested changes Dec 20, 2023
View reviewed changes
Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks fine. Guess we can see how useful all those metrics are and consider later to add more if needed....
Left a comment that should be handled.

@bx80 bx80 added the Needs Review PRs that need a code review label Dec 21, 2023
@bx80 bx80 requested a review from sgiehl December 21, 2023 03:46
michalkleiner
michalkleiner approved these changes Dec 21, 2023
View reviewed changes
Copy link
Contributor

@michalkleiner michalkleiner left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and works well, test failures unrelated.
✅ from me.

@bx80 bx80 merged commit 55f6369 into 5.x-dev Dec 28, 2023
@bx80 bx80 deleted the m21666-archiving-diagnostics branch December 28, 2023 22:39
@sgiehl sgiehl added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michalkleiner michalkleiner michalkleiner approved these changes

@sgiehl sgiehl Awaiting requested review from sgiehl

Assignees

@bx80 bx80

Labels
Easier debugging For issues that make troubleshooting issues for developers easier. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Needs Review PRs that need a code review not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Milestone
5.0.1
Development

Successfully merging this pull request may close these issues.

Add new diagnostic commands to provide archiving status information
3 participants
@bx80 @michalkleiner @sgiehl