Some mysterious precision loss going on #823
Description
After upgrading to Breeze 2.0, we have found some strange issues where we need to switch from Float to Double to get the right accuracy where we really shouldn't. Here is a somewhat minimized example:
object Test {
def main(args: Array[String]): Unit = {
import breeze.linalg._
import breeze.stats.distributions.RandBasis
val outputDim = 2
val inputDim = 4
val projection =
convert(DenseMatrix.rand(outputDim, inputDim, RandBasis.withSeed(1).gaussian) / math.sqrt(outputDim), Float)
val data = DenseMatrix.eye[Float](inputDim)
val transformed = projection * data
val corrMatrixFloat = transformed.t * transformed
val transformedDouble = convert(transformed, Double)
val corrMatrixDouble = convert(transformedDouble.t * transformedDouble, Float)
println((corrMatrixDouble - corrMatrixFloat).toString(1000, 1000))
/* prints
-2.9802322E-8 0.0 -7.4505806E-9 0.0
0.0 1.195261 1.020851 0.0
-7.4505806E-9 0.0 0.0 0.0
0.0 0.0 0.0 0.0
*/
}
}As far as I can tell, entry (2, 2) in corrMatrixFloat is just entirely wrong in a way that should not happen because of precision issues.
I'm on a 2019 Macbook Pro.