Paper: Hyper mask - projecting a talking head onto a real object

Hyper mask - projecting a talking head onto a real object

Tatsuo Yotsukura, Shigeo Morishima, Frank Nielsen, Kim Binsted and Claudio S. Pinhanez

Watch videos of hypermask: video 1 (3.5 MB) video 2 (2.5 MB)

Abstract:
HyperMask is a system which projects an animated face onto a physical mask worn by an actor. As the mask moves within a prescribed area, its position and orientation are detected by a camera and the projected image changes with respect to the viewpoint of the audience. The lips of the projected face are automatically synthesized in real time with the voice of the actor, who also controls the facial expressions. As a theatrical tool, HyperMask enables a new style of storytelling. As a prototype system, we upt a self-contained HyperMask system in a trolley (disguised as a linen cart), so that it projects onto the mask worn by the actor pushing the trolley

Key words: Talking heads, Homography, Neural networks, Computerized theatrical performances, Lip-synchronization.

Download the PDF paper here © Springer-Verlag (appeared in The Visual Computer)(407 Kb, 10 pages, 12 figures).
The original publication is available at SpringerLink.
DOI: 10.1007/s003710100140

Download the PDF paper of the Siggraph sketch here © ACM Siggraph 1999 (30 KB, 4 pages, 1 figure)
The original publication is available at ACM.

Bibtex entry:


@Article{ynbmp-hmthpro:2002,
  author = {Tatsuo Yotsukura and Frank Nielsen and Kim Binsted and Shigeo Morishima and Claudio S. Pinhanez},
  title  = {{Hypermask: Talking Head Projected onto Real Object}},
  year   = {2002},
  pages  = {111-120},
  journal = {The Visual Computer},
  volume = {18}, 
  number = {2},
  doi = {http://dx.doi.org/10.1007/s003710100140}
}

@InProceedings{pnb-pcgmssctm:1999,
 author = {Claudio Pinhanez and Frank Nielsen and Kim Binsted},
 title = {Projecting computer graphics on moving surfaces: a simple calibration and tracking method},
 booktitle = {ACM SIGGRAPH 99 Conference abstracts and applications},
 year = {1999},
 isbn = {1-58113-103-8},
 pages = {266},
 location = {Los Angeles, California, United States},
 doi = {http://doi.acm.org/10.1145/311625.312166},
 publisher = {ACM Press}
 }

Kim Binsted

Shigeo Morishima

Frank Nielsen Claudio Pinhanez

Tatsuo Yotsukura

Publications:

Tatsuo Yotsukura, Frank Nielsen, Kim Binsted, Shigeo Morishima and Claudio S. Pinhanez,
Hypermask: Talking Head Projected onto Real Object,
The Visual Computer Journal, Springer-Verlag,
Volume 18, Number 2, pp. 111-120, 2002.
Kim Binsted, Takafumi Misawa, Shigeo Morishima and Frank Nielsen,
Danger Hamster 2000,
ACM SIGGRAPH, Emerging Technologies, Conference Abstracts and Applications
page 81, 2000.
Frank Nielsen, Claudio S. Pinhanez and Kim Binsted,
Projecting Computer Graphics on Moving Surfaces: A Simple Calibration and Tracking Method
ACM SIGGRAPH, technical sketch, Conference Abstracts and Applications,
page 266, ISBN:1-58113-103-8, 1999.
Tatsuo Yotsukura, Frank Nielsen, Kim Binsted, Nobuji Tetsutani, Ryouhei Nakatsu, Shigeo Morishima,
HyperMask: Reactive Talking Head for Storytelling,
The Institute of Electronics, Information and Communication Engineers,
IEICE D-II, Volume J85-D-II(1), pp. 36-45, 2002.
Tatsuo Yotsukura, Frank Nielsen, Kim Binsted, Nobuji Tetsutani, Ryouhei Nakatsu, Shigeo Morishima,
HyperMask: Reactive Talking Head for Storytelling,
Proceedings of Eurographics,
Short presentation, pp. 305-310, 2001.
Shigeo Morishima, Tatsuo Yotsukura, Frank Nielsen, Kim Binsted and Claudio S. Pinhanez,
HyperMask: Face Image Synthesis Driven by Natural Voice and Projected onto Real Objects,
International Association of Science and Technology for Development (IASTED),
Proceedings of the IASTED International Conference, Artificial Intelligence and Soft Computing,
pp. 342-347, 2001.
Kim Binsted, Shigeo Morishima, Frank Nielsen and Tatsuo Yotsukura,
HyperMask: Projection onto 3D moving surfaces.,
NICOGRAPH, grand prize award (in japanese),
2000.
Kim Binsted, Shigeo Morishima, Frank Nielsen, Claudio S. Pinhanez and Tatsuo Yotsukura,
HyperMask: Talking Head Projected Onto Real Objects,
Proceedings of Multimedia Modeling (MMM), World Scientific,
pp. 403-412, 2000.
Kim Binsted, Shigeo Morishima, Frank Nielsen and Tatsuo Yotsukura,
HyperMask: Projection onto 3D moving surfaces.
Interact, Interactive Experiences,
1999.
Kim Binsted, Frank Nielsen and Tatsuo Yotsukura,
HyperMask: Projection onto 3D moving surfaces,
Technical Digests of the 9th Sony Research Forum,
page 33, 1999.
Kim Binsted, Frank Nielsen and Tatsuo Yotsukura,
HyperMask: Projection onto 3D moving surfaces,
Proceedings of the 9th Sony Research Forum,
pp. 225-228, ISSN 1340-3508, 1999.
Kim Binsted, Frank Nielsen, Shigeo Morishima,
HyperMask: Virtual Reactive Faces for Storytelling,
ACM SIGGRAPH, Emerging Technologies, Conference Abstracts and Applications,
page 186, 1999.

Our team T-shirt designed by Kim:

Last updated, 2003.