European Conference on Computer Vision (ECCV), 2020 We introduce a fully automatic pipeline for inferring depth, occlusion, and lighting/shadow information from image sequences of a scene. All theses information is extracted just by using people (and other objects such as cars) as scene probes to passively scan the scene. We also develop a tool for image compositing based on the inferred depth, oc