a
b
c
d
initiated from the work on invariants with Mireille Boutin and Ronan Fablet
Let us think about the process of taking a picture. Particles of light starting from3D objects travel on a straight line in the direction of the camera center, leaving their trace on a film at the intersection of the image plane and the ray of light.
It seems natural to represent images either by the trace on the image plane or the amount of light coming from all recorded directions. This second representation is based on light ray directions, rather than 2D coordinates. The two representations are strictly equivalent if we know how to relate the printed image to the physical image plane built in the camera, i.e. the internal calibration of the camera.
Rays, unlike pixels, describe not only the image but also the 3D scene that produced the image. When the camera or the scene moves, the corresponding changes on the rays are directly related to the 3D quantities. The equations involved are simple to understand, and naturally segment stationary parts of the images from moving elements.
The ray representation leads to iterative linear algorithms for structure and motion estimation. These can recover a minimal set of depth and motion parameters in Euclidean space in both cases of correspondence-based and direct methods, and integrate robust and statistical techniques.
|
|
b |
|
c |
d |
An example of correspondence-based depth and motion estimation using the light ray representation ((a) initial correspondences, (b) segmented sets of points (circle: moving points, diamond: background, square: outliers), (c) the estimated depth at the moving points (white dots are closer, dark dots are farther), (d) a 3D reconstruction of these points. The estimated motion direction is [-0.9989 0.0101 -0.0466]).
 |
 |
 |
 |
 |
 |
Acknowledgements
This work has been initiated in the SHAPE
Lab at Brown University, under the NSF grants ITR # 0205477 and KDI #BCS-9980091.
I would like to thank David Cooper, Joseph Mundy and Michael Black for their
helpful discussions and comments on this work.
Pilou