US Patent No: 6,072,496
Number of patents in Portfolio can not be more than 2000
Method and system for capturing and representing 3D geometry, color and shading of facial expressions and other animated objects
Stats
-
Jun 6, 2000
Issued date -
Jun 8, 1998
filing date -
09/093,590
serial no -
In Force
status
Importance
Loading Importance Indicators...
Overall
|
US Family Size
|
International Coverage
|
|
Patent Longevity
|
Forward Citations
|
Abstract
The method captures a 3D model of a face, which includes a 3D mesh and a series of deformations of the mesh that define changes in position of the mesh over time (e.g., for each frame). The method also builds a texture map associated with each frame in an animation sequence. The method achieves significant advantages by using markers on an actor's face to track motion of the face over time and to establish a relationship between the 3D model and texture. Specifically, videos of an actor's face with markers are captured from multiple cameras. Stereo matching is used to derive 3D locations of the markers in each frame. A 3D scan is also performed on the actor's face with the markers to produce an initial mesh with markers. The markers from the 3D scan are matched with the 3D locations of the markers in each frame from the stereo matching process. The method determines how the position of the mesh changes from frame to frame by matching the 3D locations of the markers from one frame to the next. The method derives textures for each frame by removing the dots from the video data, finding a mapping between texture space and the 3D space of the mesh, and combining the camera views for each frame into a signal texture map. The data needed to represent facial animation includes: 1) an initial 3D mesh, 2) 3D deformations of the mesh per frame, and 3) a texture map associated with each deformation. The method compresses 3D geometry by decomposing the deformation data into basis vectors and coefficients. The method compresses the textures using video compression.
Loading the Abstract Image...
First Claim
Related Publications
Loading Related Publications...
- 15 United States
- 10 France
- 8 Japan
- 7 China
- 5 Korea
- 2 Other
Patent Owner(s)
| Patent Owner | Address | Total Patents |
|---|---|---|
| MICROSOFT CORPORATION | REDMOND, WA | 24226 |
International Classification(s)
- [Classification Symbol]
- [Patents Count]
Inventor(s)
| Inventor Name | Address | # of filed Patents | Total Citations |
|---|---|---|---|
| Grimm, Cindy Marie | Seattle, WA | 1 | 174 |
| Guenter, Brian | Redmond, WA | 2 | 193 |
| Malvar, Henrique Sarmento | Sammamish, WA | 8 | 340 |
Cited Art
| Patent Info | (Count) | # Cites | Year |
|---|---|---|---|
|
|
|||
| 5,675,720 Method of searching for points of closest approach, and preprocessing method therefor | 39 | 1996 | |
|
|
|||
| 5,524,187 Worlds-within-worlds nested display and interaction system and method | 30 | 1994 | |
Patent Citation Ranking
Forward Cites
| Patent Info | (Count) | # Cites | Year |
|---|---|---|---|
|
|
|||
| 6,271,847 Inverse texture mapping using weighted pyramid blending and view-dependent weight maps | 65 | 1998 | |
| 6,275,235 High precision texture wrapping method and device | 42 | 1998 | |
| 6,487,565 Updating animated images represented by scene graphs | 31 | 1998 | |
| 6,614,428 Compression of animated geometry using a hierarchical level of detail coder | 29 | 1999 | |
| 7,020,305 System and method providing improved head motion estimations for animation | 10 | 2000 | |
| 6,807,290 Rapid computer modeling of faces for animation | 47 | 2001 | |
| 6,771,303 Video-teleconferencing system with eye-gaze correction | 65 | 2002 | |
| 7,392,472 Layout analysis | 7 | 2002 | |
| 7,263,227 Activity detector | 5 | 2002 | |
| 7,164,797 Clustering | 1 | 2002 | |
| 7,515,173 Head pose tracking system | 13 | 2002 | |
| 7,120,297 Segmented layered image system | 23 | 2002 | |
| 7,110,596 System and method facilitating document image compression utilizing a mask | 4 | 2002 | |
| 7,043,079 "Don't care" pixel interpolation | 3 | 2002 | |
| 7,024,039 Block retouching | 5 | 2002 | |
| 7,428,330 Cyclopean virtual imaging via generalized probabilistic smoothing | 13 | 2003 | |
| 7,809,183 Gaze manipulation | 2 | 2003 | |
| 7,570,803 Virtual camera translation | 4 | 2004 | |
| 7,292,735 Virtual image artifact detection | 2 | 2004 | |
| 7,257,272 Virtual image generation | 7 | 2004 | |
| 6,980,671 Rapid computer modeling of faces for animation | 4 | 2004 | |
| 6,950,537 Rapid computer modeling of faces for animation | 2 | 2004 | |
| 6,944,320 Rapid computer modeling of faces for animation | 6 | 2004 | |
| 7,986,372 Systems and methods for smart media content thumbnail extraction | 1 | 2004 | |
| 7,706,575 System and method providing improved head motion estimations for animation | 0 | 2004 | |
| 7,892,079 Detect-point-click (DPC) based gaming systems and techniques | 0 | 2004 | |
| 7,174,035 Rapid computer modeling of faces for animation | 1 | 2004 | |
| 7,149,329 Rapid computer modeling of faces for animation | 1 | 2004 | |
| 7,039,219 System and method providing improved head motion estimations for animation | 6 | 2004 | |
| 7,082,212 Rapid computer modeling of faces for animation | 62 | 2005 | |
| 6,993,163 Rapid computer modeling of faces for animation | 5 | 2005 | |
| 7,065,233 Rapid computer modeling of faces for animation | 2 | 2005 | |
| 7,181,051 Rapid computer modeling of faces for animation | 6 | 2005 | |
| 7,158,658 Rapid computer modeling of faces for animation | 5 | 2005 | |
| 7,376,275 Clustering | 5 | 2005 | |
| 8,196,032 Template-based multimedia authoring and sharing | 0 | 2005 | |
| 7,397,952 "Don't care" pixel interpolation | 3 | 2005 | |
| 7,142,698 Rapid computer modeling of faces for animation | 3 | 2005 | |
| 7,212,656 Rapid computer modeling of faces for animation | 8 | 2006 | |
| 7,149,330 Rapid computer modeling of faces for animation | 1 | 2006 | |
| 7,133,540 Rapid computer modeling of faces for animation | 7 | 2006 | |
| 7,512,274 Block retouching | 1 | 2006 | |
| 8,098,730 Generating a motion attention model | 1 | 2006 | |
| 8,180,826 Media sharing and authoring on the web | 0 | 2006 | |
| 7,764,834 System and method facilitating document image compression utilizing a mask | 1 | 2006 | |
| 7,376,266 Segmented layered image system | 9 | 2006 | |
| 7,386,171 Activity detector | 0 | 2007 | |
|
|
|||
| 6,563,503 Object modeling for computer simulation and animation | 42 | 1999 | |
| 6,717,577 Vertex cache for 3D computer graphics | 24 | 1999 | |
| 7,119,813 Variable bit field encoding | 3 | 2000 | |
| 7,538,772 Graphics processing system with enhanced memory controller | 4 | 2000 | |
| 7,196,710 Method and apparatus for buffering graphics data in a graphics system | 12 | 2000 | |
| 7,184,059 Graphics system with copy out conversions between embedded frame buffer and main memory | 32 | 2000 | |
| 7,061,502 Method and apparatus for providing logical combination of N alpha operations within a graphics system | 3 | 2000 | |
| 7,034,828 Recirculating shade tree blender for a graphics system | 36 | 2000 | |
| 7,002,591 Method and apparatus for interleaved processing of direct and indirect texture coordinates in a graphics system | 55 | 2000 | |
| 6,867,781 Graphics pipeline token synchronization | 30 | 2000 | |
| 6,811,489 Controller interface for a graphics system | 37 | 2000 | |
| 6,707,458 Method and apparatus for texture tiling in a graphics system | 36 | 2000 | |
| 6,700,586 Low cost graphics with stitching processing hardware support for skeletal animation | 18 | 2000 | |
| 6,636,214 Method and apparatus for dynamically reconfiguring the order of hidden surface processing based on rendering mode | 73 | 2000 | |
| 6,618,048 3D graphics rendering system for performing Z value clamping in near-Z range to maximize scene resolution of visually important Z components | 63 | 2000 | |
| 7,205,999 Method and apparatus for environment-mapped bump-mapping in a graphics system | 4 | 2004 | |
| 7,075,545 Graphics system with embedded frame buffer having reconfigurable pixel formats | 3 | 2005 | |
| 7,307,640 Method and apparatus for efficient generation of texture coordinate displacements for implementing emboss-style bump mapping in a graphics rendering system | 2 | 2005 | |
| 7,307,638 Method and apparatus for interleaved processing of direct and indirect texture coordinates in a graphics system | 12 | 2005 | |
| 7,176,919 Recirculating shade tree blender for a graphics system | 26 | 2005 | |
| 7,576,748 Graphics system with embedded frame butter having reconfigurable pixel formats | 0 | 2006 | |
| 7,317,459 Graphics system with copy out conversions between embedded frame buffer and main memory for producing a streaming video image as a texture on a displayed object image | 3 | 2006 | |
| 7,701,461 Method and apparatus for buffering graphics data in a graphics system | 2 | 2007 | |
| 8,098,255 Graphics processing system with enhanced memory controller | 0 | 2009 | |
| 7,995,069 Graphics system with embedded frame buffer having reconfigurable pixel formats | 0 | 2009 | |
|
|
|||
| 6,900,809 Method and apparatus for animation of an object on a display | 0 | 2001 | |
| 8,154,612 Systems, methods, and apparatus for image processing, for color classification, and for skin color detection | 4 | 2005 | |
| 7,379,566 Optical flow based tilt sensor | 69 | 2006 | |
| 7,430,312 Creating 3D images of objects by illuminating with infrared patterns | 38 | 2006 | |
| 8,144,118 Motion-based tracking | 0 | 2006 | |
| 7,827,698 Orientation-sensitive signal output | 6 | 2008 | |
| 7,570,805 Creating 3D images of objects by illuminating with infrared patterns | 36 | 2008 | |
| 7,822,267 Enhanced object reconstruction | 0 | 2008 | |
| 7,953,271 Enhanced object reconstruction | 1 | 2010 | |
| 8,015,718 Orientation-sensitive signal output | 1 | 2010 | |
| 8,213,686 Optical flow based tilt sensor | 0 | 2010 | |
| 8,218,858 Enhanced object reconstruction | 0 | 2011 | |
| 8,230,610 Orientation-sensitive signal output | 0 | 2011 | |
|
|
|||
| 7,123,263 Automatic 3D modeling system and method | 11 | 2002 | |
| 7,355,607 Automatic 3D modeling system and method | 10 | 2006 | |
| 7,952,577 Automatic 3D modeling system and method | 3 | 2008 | |
| 8,094,150 Automatic 3D modeling system and method | 1 | 2011 | |
| 8,289,317 Automatic 3D modeling | 0 | 2011 | |
|
|
|||
| 7,848,564 Three-dimensional motion capture | 0 | 2006 | |
| 8,199,152 Combining multiple session content for animation libraries | 0 | 2007 | |
| 8,130,225 Using animation libraries for object identification | 0 | 2007 | |
| 8,144,153 Model production for animation libraries | 0 | 2007 | |
| 8,019,137 Three-dimensional motion capture | 3 | 2009 | |
|
|
|||
| 6,678,413 System and method for object identification and behavior characterization using video analysis | 99 | 2000 | |
| 7,209,588 Unified system and method for animal behavior characterization in home cages using video analysis | 4 | 2003 | |
| 7,643,655 System and method for animal seizure detection and classification using video analysis | 7 | 2006 | |
| 7,817,824 Unified system and method for animal behavior characterization from top view using video analysis | 0 | 2009 | |
|
|
|||
| 6,539,099 System and method for visual chat | 12 | 1999 | |
| 6,876,754 System and method for visual chat | 4 | 2003 | |
| 7,167,577 System and method for visual chat | 1 | 2005 | |
| 7,388,966 System and method for visual chat | 0 | 2006 | |
|
|
|||
| 7,050,655 Method for generating an animated three-dimensional video head | 4 | 2001 | |
| 7,283,687 Imaging for virtual cameras | 3 | 2001 | |
| 8,024,316 Providing content using stored query information | 1 | 2008 | |
| 8,341,138 Providing content using stored query information | 0 | 2011 | |
|
|
|||
| 7,961,947 FACS cleaning in motion capture | 2 | 2007 | |
| 7,733,346 FACS solving in motion capture | 0 | 2007 | |
| 8,330,823 Capturing surface in motion picture | 0 | 2007 | |
|
|
|||
| 6,661,907 Face detection in digital images | 97 | 1999 | |
| 7,218,759 Face detection in digital images | 78 | 1999 | |
|
|
|||
| 7,718,105 Fabricating multi-component skin systems for robotics and other applications | 2 | 2008 | |
| 7,887,729 Fabricating multi-component skin systems for robotics and other applications | 0 | 2010 | |
|
|
|||
| 7,200,642 Method and apparatus for electronic delivery of electronic model images | 4 | 2001 | |
| 7,215,803 Method and apparatus for interactive remote viewing and collaboration of dental images | 17 | 2004 | |
|
|
|||
| 6,304,333 Apparatus and method of performing dithering in a simplex in color space | 5 | 1998 | |
| 7,529,418 Geometry and view assisted transmission of graphics image streams | 1 | 2004 | |
|
|
|||
| 7,830,384 Animating graphical objects using input video | 0 | 2006 | |
| 7,768,528 Replacement of faces in existing video | 1 | 2006 | |
|
|
|||
| 7,224,729 Method of progressively coding/decoding 3-D mesh information and apparatus thereof | 3 | 1999 | |
| 7,928,978 Method for generating multi-resolution three-dimensional model | 0 | 2007 | |
|
|
|||
| 8,219,438 Method and system for measuring shopper response to products based on behavior and facial expression | 1 | 2008 | |
| 8,401,248 Method and system for measuring emotional and attentional response to dynamic digital media content | 0 | 2008 | |
|
|
|||
| 7,016,824 Interactive try-on platform for eyeglasses | 7 | 2001 | |
|
|
|||
| 6,853,373 Methods, apparatus and computer program products for modeling three-dimensional colored objects | 14 | 2001 | |
|
|
|||
| 6,525,731 Dynamic view-dependent texture mapping | 19 | 1999 | |
|
|
|||
| 6,816,159 Incorporating a personalized wireframe image in a computer software application | 8 | 2001 | |
|
|
|||
| 6,654,018 Audio-visual selection process for the synthesis of photo-realistic talking-head animations | 24 | 2001 | |
|
|
|||
| 6,967,658 Non-linear morphing of faces and their dynamics | 9 | 2001 | |
|
|
|||
| 6,553,337 Parameterization of subdivision surfaces | 11 | 1998 | |
|
|
|||
| 6,909,440 Stepped-decay video morphing for liquid crystal displays | 1 | 2000 | |
|
|
|||
| 6,650,335 Method for extracting static and dynamic super-resolution textures form a sequence of images | 2 | 2001 | |
|
|
|||
| 7,184,049 Image processing method and system | 3 | 2003 | |
|
|
|||
| 8,300,046 Attraction system and attraction providing method | 0 | 2011 | |
|
|
|||
| 7,675,520 System, method and computer program for creating two dimensional (2D) or three dimensional (3D) computer animation from video | 3 | 2006 | |
|
|
|||
| 7,978,191 System and method for locating anatomies of interest in a 3D volume | 1 | 2007 | |
|
|
|||
| 6,728,417 Measurement apparatus | 11 | 1999 | |
|
|
|||
| 7,978,886 System and method for anatomy based reconstruction | 0 | 2005 | |
|
|
|||
| 7,991,203 Video communication device and image processing system and method of the same | 0 | 2008 | |
|
|
|||
| 7,113,848 Human emulation robot system | 12 | 2004 | |
|
|
|||
| 7,103,211 Method and apparatus for generating 3D face models from one camera | 13 | 2002 | |
|
|
|||
| 7,050,085 System and method for camera calibration | 3 | 2000 | |
|
|
|||
| 7,245,305 Shading of images using texture | 1 | 2005 | |
|
|
|||
| 6,478,680 Game apparatus, method of displaying moving picture, and game program product | 22 | 2000 | |
|
|
|||
| 7,035,433 Image recognition method and apparatus | 4 | 1999 | |
|
|
|||
| 7,034,832 Computer readable medium storing 3-D image processing program, 3-D image processing method and device, 3-D image processing program, and video game device | 1 | 2002 | |
|
|
|||
| 6,466,700 Method of processing a noisy multidimensional image and medical imaging apparatus for carrying out this method | 9 | 1998 | |
|
|
|||
| 6,486,881 Basis functions of three-dimensional models for compression, transformation and streaming | 3 | 2001 | |
|
|
|||
| 7,710,452 Remote video monitoring of non-urban outdoor sites | 0 | 2006 | |
|
|
|||
| 6,330,281 Model-based view extrapolation for interactive virtual reality systems | 10 | 2000 | |
|
|
|||
| 7,015,934 Image displaying apparatus | 0 | 2001 | |
|
|
|||
| 6,885,761 Method and device for generating a person's portrait, method and device for communications, and computer product | 21 | 2001 | |
|
|
|||
| 7,388,969 Method of measuring rotational amount of body having curved surface and direction of rotational axis thereof, apparatus of measuring rotational amount thereof and direction of rotational axis thereof, and method of specifying three-dimensional posture thereof | 0 | 2003 | |
|
|
|||
| 7,391,907 Spurious object detection in a video surveillance system | 11 | 2004 | |
|
|
|||
| 8,207,963 System and method for performing motion capture and image reconstruction | 0 | 2007 | |
|
|
|||
| 8,139,076 System, method, and program for determining luminance values of multiple polygons which represent a surface of a subject | 0 | 2010 | |
|
|
|||
| 6,608,631 Method, apparatus, and computer program product for geometric warps and deformations | 75 | 2000 | |
|
|
|||
| 7,956,890 Adaptive multi-modal integrated biometric identification detection and surveillance systems | 2 | 2005 | |
|
|
|||
| 7,136,524 Robust perceptual color identification | 6 | 2005 | |
|
|
|||
| 7,990,384 Audio-visual selection process for the synthesis of photo-realistic talking-head animations | 0 | 2003 | |
|
|
|||
| 6,873,324 Data processing method, recording medium and data processing apparatus | 8 | 2001 | |
|
|
|||
| 7,024,052 Motion image decoding apparatus and method reducing error accumulation and hence image degradation | 6 | 2002 | |
|
|
|||
| 8,115,774 Application of selective regions of a normal map based on joint position in a three-dimensional model | 1 | 2006 | |
|
|
|||
| 7,764,828 Method, apparatus, and computer program for processing image | 5 | 2005 | |
|
|
|||
| 6,577,308 Data processing method and apparatus and information furnishing medium | 4 | 1999 | |
|
|
|||
| 7,573,952 Barycentric coordinate technique for resampling quantized signals | 4 | 2005 | |
|
|
|||
| 6,753,863 System and method for streaming real time animation data file | 4 | 2000 | |
|
|
|||
| 6,680,735 Method for correcting gradients of irregular spaced graphic data | 16 | 2000 | |
|
|
|||
| 8,059,917 3-D modeling | 1 | 2007 | |
|
|
|||
| 8,339,402 System and method of producing an animated performance utilizing multiple cameras | 0 | 2007 | |
|
|
|||
| 7,853,876 Method to imitate lifelike images for computer deformed objects | 1 | 2007 | |
|
|
|||
| 8,284,202 Methods and apparatus for capturing and rendering dynamic surface deformations in human motion | 0 | 2007 | |
|
|
|||
| 6,366,684 Image processing method and system involving contour detection steps | 16 | 1999 | |
Maintenance Fees
| Fee | Large entity fee | small entity fee | micro entity fee | due date |
|---|
| Fee | Large entity fee | small entity fee | micro entity fee |
|---|---|---|---|
| Surcharge after expiration - Late payment is unavoidable | $700.00 | $350.00 | $175.00 |
| Surcharge after expiration - Late payment is unintentional | $1,640.00 | $820.00 | $410.00 |