As stated, since colmap fails to register for Lytro style data, I am trying to manually generate poses and mpi_bds. However, I think I might do something wrong here.

I figured out that the poses is a 5x3 matrix which generates the homography.
If the horizontal baseline and vertical baseline of each view is x and y and the image is of size hxw and the focus is f, is this the right matrix for poses?

1 0 0 (-y) h
0 1 0 (-x) w
0 0 1 0    f