From the source code it looks like once you've made a request to read contents of the file it will set the objectStat field and will return that instead of making another request. However, when you call Stat() on the object it makes another request. Debugging showed that both isStarted and objectInfoSet are false:

And the reason is that response.Error is io.EOF and the other parts of the code never get executed.

obj, err := client.GetObject(ctx, "my-bucket", "my-file.txt", minio.GetObjectOptions{})
if err != nil {
	return err
}

buf, err := io.ReadAll(obj)
lockfileStat, err := lockfile.Stat() // <--- this makes another request
if err != nil {
	return err
}