I could be wrong but I think the scale calculation for the AutoMultivariateNormal is incorrect. According to
https://docs.pyro.ai/en/stable/_modules/pyro/infer/autoguide/guides.html#AutoMultivariateNormal
it is self.scale * self.scale_tril.diag(). However, the scale should be the square root of the diagonal of the covariance, which would be something more like (self.scale[..., None] * self.scale_tril)**2).sum(-1).sqrt(). Note these are not equivalent since the latter depends on the off-diagonal elements of scale_tril whereas the former does not.