Referrer Websites report should group same label (HTTP vs. HTTPS) #13882
Description
While reviewing visits from Tumblr, I expanded referrals and clicked "open row evolution" on the first sub-row:
To my surprise it showed a very sparse graph with only a few views each day. I found that whichever of the two rows I clicked evolution on, it was giving me the data for the second row. Hovering over the 'index' links to the left, I found that the first row was for referrals from the HTTPS url, and the second for HTTP.
The parameters encoded within the graph's request URL are:
date=today
apiMethod=Referrers.getWebsites
label=t.umblr.com+>+@index
disableLink=1
module=CoreHome
action=getRowEvolutionPopover
colors={"backgroundColor":"#ffffff","lineColor":"#162c4a","minPointColor":"#ff7f7f","maxPointColor":"#75bf7c","lastPointColor":"#55aaff","fillColor":"#ffffff"}
flat=0
idSite=1
period=day
These requests are the same if I click on either row evolution icon - which makes sense, the path just says "index". But when processing the request, it picked the less-used option - not the option I selected. If it's going to separate them by scheme/URL, duplicating the label, it should use scheme/URL for the evolution.
"Metrics for Website" on the popup itself showed the website domain name as well as the path, but not the access scheme. Perhaps if it had shown "http:" or "https:" as well, this issue would have been more visible.
There seems to be plenty of information within the row's to make this distinction, e.g.:
<tr data-segment-filter="referrerUrl==https%3A%2F%2Ft.umblr.com%2F" data-url-label="https://t.umblr.com/" data-row-metadata="{"segment":"referrerUrl==https%3A%2F%2Ft.umblr.com%2F","url":"https:\/\/t.umblr.com\/"}" class=" ">
In this particular case it doesn't matter because I can just use the top-level graph which does work as it combines views from all paths (and here, there is only one path). However, in the general case it is a problem:
- HTTP or HTTPS? Can't tell at a glance. Row evolution for both picks the second.
I'm using Matamo 3.8.0-b5 on nginx with PHP 7.3 and MariaDB 10.1.37 on Debian Stretch.
Even more generally (maybe this has an issue already): HTTPS and HTTP requests are not combined. While technically these may be completely different sites, in practice they're almost certainly not and would be more useful together. One of our main referrers is, by default, HTTP, but may be configured to force a user's requests to HTTPS. So some visitors from it will be using HTTP and others HTTPS, resulting in two entries.
This makes it difficult to track unique requests from that site (you have to either open them twice or try to remember which complicated numerical URLs you've seen that day) and also decreases the number of useful samples in the top-50 AJAX report available via the dashboard.
Perhaps there should be an option - and I'd argue that it should be the default - to use 'group by' or similar in the database requests to merge entries which will end up with the same label. Of course the same option would have to be used throughout where requests are made, like row evolution. Or the merger could be done in PHP but this would be less efficient and I imagine it wouldn't end up with 50 items in top-50 etc.