Digital Matting and Compositing

Matting and compositing are important operations in the production of special effects. These techniques enable directors to embed actors in a world that exists only in imagination, or to revive creatures that have been extinct for millions of years. During matting, foreground elements are extracted from a film or video sequence. During compositing, the extracted foreground elements are placed over novel background images.

Traditional approaches to matting include blue-screen matting and rotoscoping. The former requires filming in front of an expensive blue screen under carefully controlled lighting, and the latter demands talent and intensive user interaction. Our Bayesian matting algorithm (CVPR 2001) can pull alpha mattes of complex shapes from natural images. In the "video matting" paper (SIGGRAPH 2002), we extended this approach to video by interpolating user-drawn keyframes using optical flow. A novel technique for smoke matte extraction is also demonstrated.

Traditional compositing operations can only model color blending effects like anti-aliasing, motion blur and transparency. This model, however, can't model reflections, refractions and shadows. In SIGGRAPH 1999, we introduced environment matting which can captures how a foreground object refracts and reflects light. The foreground object can then be placed in a new environment using environment compositing, where it will refract and reflect light from that scene. We later developed more sophisticated sampling schemes to capture mattes with higher accuracies, and techniques requiring fewer images, to allow for real-time capture. Shadows are yet another effects that traditional approaches fail to model correctly. In SIGGRAPH 2003, we introduced a novel process called "shadow matting and compositing" to acquire the photometric and geometric properties of the background for making realistic shadow composites.

• Aseem Agarwala - UW
• Yung-Yu Chuang - UW
• Brian Curless - UW
• Dan Goldman - UW/Industrial Light and Magic
• Joel Hindorff - UW (Applied Math)
• David Salesin - UW/Microsoft Research
• Rick Szeliski - Microsoft Research
• Dawn Werner - UW
• Doug Zongker - UW

Shadow Matting and Compositing

Yung-Yu Chuang, Dan B Goldman, Brian Curless, David Salesin, Richard Szeliski

Proc. ACM SIGGRAPH 2003

Video Matting of Complex Scenes

Yung-Yu Chuang, Aseem Agarwala, Brian Curless, David Salesin, Richard Szeliski

Proc. ACM SIGGRAPH 2002

A Bayesian Approach to Digital Matting

Yung-Yu Chuang, Brian Curless, David Salesin, Richard Szeliski

Proc. CVPR 2001

Environment Matting Extensions: Towards Higher Accuracy and Real-Time Capture

Yung-Yu Chuang, Douglas Zongker, Joel Hindorff, Brian Curless, David Salesin, Richard Szeliski

Proc. ACM SIGGRAPH 2000

Environment Matting and Compositing

Douglas Zongker, Dawn Werner, Brian Curless, David Salesin

Proc. ACM SIGGRAPH 1999

• NSF grants 9553199, 9803226, CCR-987365 and DMS-9803226
• University of Washington Animation Research Labs
• Industrial gifts from Intel, Microsoft and Pixar