Digital Matting and Compositing

Digital Matting and Compositing


Matting and compositing are important operations in the production of special effects. These techniques enable directors to embed actors in a world that exists only in imagination, or to revive creatures that have been extinct for millions of years. During matting, foreground elements are extracted from a film or video sequence. During compositing, the extracted foreground elements are placed over novel background images.

Traditional approaches to matting include blue-screen matting and rotoscoping. The former requires filming in front of an expensive blue screen under carefully controlled lighting, and the latter demands talent and intensive user interaction. Our Bayesian matting algorithm (CVPR 2001) can pull alpha mattes of complex shapes from natural images. In the "video matting" paper (SIGGRAPH 2002), we extended this approach to video by interpolating user-drawn keyframes using optical flow. A novel technique for smoke matte extraction is also demonstrated.

Traditional compositing operations can only model color blending effects like anti-aliasing, motion blur and transparency. This model, however, can't model reflections, refractions and shadows. In SIGGRAPH 1999, we introduced environment matting which can captures how a foreground object refracts and reflects light. The foreground object can then be placed in a new environment using environment compositing, where it will refract and reflect light from that scene. We later developed more sophisticated sampling schemes to capture mattes with higher accuracies, and techniques requiring fewer images, to allow for real-time capture. Shadows are yet another effects that traditional approaches fail to model correctly. In SIGGRAPH 2003, we introduced a novel process called "shadow matting and compositing" to acquire the photometric and geometric properties of the background for making realistic shadow composites.




This research is supported by:
  • NSF grants 9553199, 9803226, CCR-987365 and DMS-9803226
  • University of Washington Animation Research Labs
  • Industrial gifts from Intel, Microsoft and Pixar

Copyright © Since 1996 NTU CMLab All Rights Reserved.