Intrinsic image decomposition from multiple photographs

Abstract

Editing materials and lighting is a common image manipulation task that requires significantexpertise to achieve plausible results. Each pixel aggregates the effect of both materialand lighting, therefore standard color manipulations are likely to affect both components.Intrinsic image decomposition separates a photograph into independent layers: reflectance,which represents the color of the materials, and illumination, which encodes the effect oflighting at each pixel.In this thesis, we tackle this ill-posed problem by leveraging additional information providedby multiple photographs of the scene. We combine image-guided algorithms withsparse 3D information reconstructed from multi-view stereo, in order to constrain the decomposition.We first present an approach to decompose images of outdoor scenes, using photographscaptured at a single time of day. This method not only separates reflectance from illumination,but also decomposes the illumination into sun, sky, and indirect layers. We thendevelop a methodology to extract lighting information about a scene solely from a few images,thus simplifying the capture and calibration steps of our intrinsic decomposition. In athird part, we focus on image collections gathered from photo-sharing websites or capturedwith a moving light source. We exploit the variations of lighting to process complex sceneswithout user assistance, nor precise and complete geometry.The methods described in this thesis enable advanced image manipulations such aslighting-aware editing, insertion of virtual objects, and image-based illumination transferbetween photographs of a collection.