researchers from MIT and Rice University have created a computer vision technique that leverages reflections to image the world. Their method uses reflections to turn glossy objects into “cameras,” enabling a user to see the world as if they were looking through the “lenses” of everyday objects like a ceramic coffee mug or a metallic paper weight.
Using images of an object taken from different angles, the technique converts the surface of that object into a virtual sensor which captures reflections. The AI system maps these reflections in a way that enables it to estimate depth in the scene and capture novel views that would only be visible from the object’s perspective. One could use this technique to see around corners or beyond objects that block the observer’s view.
This method could be especially useful in autonomous vehicles. For instance, it could enable a self-driving car to use reflections from objects it passes, like lamp posts or buildings, to see around a parked truck.
“We have shown that any surface can be converted into a sensor with this formulation that converts objects into virtual pixels and virtual sensors. This can be applied in many different areas,” says Kushagra Tiwary, a graduate student in the Camera Culture Group at the Media Lab and co-lead author of a paper on this research.