As requested by Matt herewith a bug report:

I have an '& amp ;' and an '& # 039 ;' in my website name. (written with spaces here because otherwise github shows them correctly)
On the Piwik "All websites dashboard" this look just fine as the & and ' symbols.

However in the selected website name field on the main dashboard is shows this just as '&' and ''' which looks very strange.

Also in the Piwik Reports email is looks wrong, as well in the description as in the body except for the "Hello,Please find below your weekly report for......" line that one shows correctly

Don't know if this is a bug or so, I couldn't find anything about this.
I know this issue is there since very early in v2.xx versions.

I run Piwik on my own server.

Hope it will be fix in one of the next versions, or that someone tells me how to fix (.... not by removing those characters).