"Interreflections in Computer Vision: Importance, Modeling & Application in Spec-tral Estimation" by Damien Muselet

Seminar by Damien Muselet

Abstract

The interreflection phenomenon happens whenever a scene containing concave surfaces is illuminated. Interreflections create particular color variations in the area of the scene corresponding to the concave object. Our research focused on studying the importance of these color variations happening in concave Lambertian surfaces in spectral estimation from RGB images. First, a model built on radiometric basis taking into consideration infinite number of bounces of interreflection is provided. This model relates raw RGB values with the spectral reflectance and the geometry of the surface, the spectral power distribution of the light, and the sensors’ spectral responses. Then, this model is exploited in the estimation of the spectral reflectance of folded uniformly colored surfaces from a single RGB image under a known lighting. On simulated data, our results indicate that spectral reflectance is as accurately predicted by using the interreflection approach with one light as by using approaches based on several lightings. In addition, the approach can be used in non-controlled scenario under sunlight while the intensity of light is not measured and the camera is not calibrated for the acquisition settings. The use of interreflections was also studied in the estimation of sensor responses curves form 3D color charts.

Later, a convolutional neural network was trained on datasets built from synthetic images simulated using the interreflection model. This network is used to solve the inverse problem of surface spectral reflectance estimation form a single RGB image of interreflection under unknown lighting. Experiments on simulated data showed that our approach even under an unknown illuminant outperforms the state of the art learning-based spectral reflectance estimation approaches trained for a specific known lighting. In addition, real data results showed that our method gets a better accuracy of spectral reflectance estimation under unknown lighting than our physics-based approach under a light with a known SPD.

This seminar will be held in english