Skip to content

[LookerStudio] add new metricTypes report metadata (4.x) #20425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 10, 2023

Conversation

diosmosis
Copy link
Member

Description:

This PR adds new report metadata for the upcoming official looker studio integration. Looker studio requires connectors to provide structural metadata about the data the connectors query. This metadata is used to determine how the data is displayed and aggregated (within looker studio).

Matomo reports do not currently provide this information, so it needs to be added for the upcoming connector to use. Changes include:

  • A new method in Metrics: getDefaultMetricSemanticTypes() that behaves like getDefaultMetricTranslations() and other methods.
  • A new event Metrics.getDefaultMetricSemanticTypes that allows plugins to set semantic types.
  • Add semantic types for existing core plugins.

Note on terminology: the term "semantic type" is used by looker studio and I thought it was worth using here as well, since we already a column type (the database column types). Looker studio's docs on this are here: https://developers.google.com/looker-studio/connector/reference#semantictype

Originally I tried providing some simpler default logic based on the metric/record naming conventions matomo uses, but these conventions are not applied universally, and the number of exceptions made it too complicated. (For pre-5.0 Matomo versions, the connector will try to deduce this information from metric name, but this will be error prone compared to using explicit metadata.)

Review

@diosmosis diosmosis added the Needs Review PRs that need a code review label Mar 7, 2023
@diosmosis diosmosis added this to the 4.13.4 milestone Mar 7, 2023
Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left two comments with some more general thoughts. Otherwise some Integration tests are failing and need to be fixed.

* @return string[] maps metric name => semantic type
* @api
*/
public function getMetricSemanticTypes()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be awesome to add (return) type hints to this and all other new methods were possible.

Suggested change
public function getMetricSemanticTypes()
public function getMetricSemanticTypes(): array

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sgiehl parameter type hints as well or just return type hints?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameter type hints as well, as long as they are supported by PHP 7.2 🙂

'nb_uniq_outlinks' => Metric::SEMANTIC_TYPE_NUMBER,
'nb_searches' => Metric::SEMANTIC_TYPE_NUMBER,
'nb_keywords' => Metric::SEMANTIC_TYPE_NUMBER,
'avg_time_generation' => Metric::SEMANTIC_TYPE_DURATION, // TODO: should all have the same unit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for adding this here and not directly in the metric class? Wouldn't it be useful to use the metric classes to define the semantic type where possible?
At some point in the future I would actually love to have metric classes for all available metrics and that reports need to use a metric class for each metric they contain. But that might be a bit bigger refactoring, so not sure when we will get there. But already moving everything to the Metric class that can be defined there might already be helpful....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most reports reference metrics by their string names, not by a Metric instance, so it would be difficult in Report.php to get that information. There is a getSemanticType() method in Metric though, but right now it's primarily used for ProcessedMetrics, which are currently used in Report::$processedMetrics.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. But isn't avg_time_generation a processed metric as well? But might be not that import for now. Just wanted to mention that we should try to move as much as possible to the metrics, so we have less work when we start refactoring that at some point...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just put the same metrics from the 'Metrics.getDefaultMetricTranslations' hook into the new semantic types hook. I don't expect that would hinder the move from string metric names to metric classes in Report classes in any way. The work should take hours at most regardless.

@diosmosis
Copy link
Member Author

@sgiehl updated

Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now. Will merge the PRs in submodules first

@sgiehl sgiehl force-pushed the 4.x-column-type-report-metadata branch from 99e1e88 to ac9498e Compare March 10, 2023 11:44
@diosmosis
Copy link
Member Author

@sgiehl fyi, I think there are some changes in the bandwidth pr in the branch https://github.com/matomo-org/plugin-Bandwidth/tree/20425-fix-build. I recreated this pr but forgot that one. Apologies.

@sgiehl sgiehl force-pushed the 4.x-column-type-report-metadata branch from a85b53e to 0df64ca Compare March 10, 2023 12:39
@sgiehl sgiehl merged commit e83400c into 4.x-dev Mar 10, 2023
@sgiehl sgiehl deleted the 4.x-column-type-report-metadata branch March 10, 2023 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review
Development

Successfully merging this pull request may close these issues.

2 participants