TNT logo LUH TNT

Statistical 3D Face Model

TNT members involved in this project:
Hanno Ackermann, Ph.D.
Stella Graßhof, M.Sc.
Felix Kuhnke, M.Sc.
Prof. Dr.-Ing. Jörn Ostermann

On the one hand, bodys and faces of humans differ largely in their outer appearance between individuals. However on the other hand, the wide variety of performed movements may just look alike between different persons. Therefore to describe and disentangle the two different parts, i.e. individual shape and motion of humans, is a very challenging problem.
The goal of this project is to build a versatile 3D face model, with the following properties:

  • Parameters for person and expression
  • Changing one parameter must not influence aspects of the other.
  • must be 3D

 

The proposed model is build upon a database of 2500 3D face scans. The BU3DFE database consists of 3D face scans of 100 persons, who each perform 6 emotions (anger, disgust, fear, happiness, sadness, surprise) in 4 different expression levels and the neutral expression, leading to 25 expression in total.

Firstly, the data is preprocessed, and dense correspondences between the 3D face scans are computed by a nonrigid 3D registration algorithm. Now that all data sets have the same number of points, they are arranged into a multidimensional array, i.e. tensor, and the mean shape is subtracted from each.
Using the Higher-Order Singular Value Decomposition (HOSVD), the resulting tensor is factorized into three modes: points, person and expression.

  • Face Parameter Estimation
  • 3D Approximation
  • 3D Reconstruction from Sparse 2D Landmarks
  • Analysis and Synthesis of Faces
  • Person and Expression Transfer
  • Expression Classification

 

candide3

 

tensormodel

Reconstructed 3D face with Candide3 face model

 

Reconstructed 3D face with tensor face model

 

You can find the latest code of this project on github.

If you are interested in a thesis within this project, please contact Stella Graßhof or Felix Kuhnke.

  • Conference Contributions
    • Stella Graßhof, Hanno Ackermann, Felix Kuhnke, Jörn Ostermann, Sami Brandt
      Projective Structure from Facial Motion
      15th IAPR International Conference on Machine Vision Applications (MVA) (accepted), Nagoya (Japan), May 2017
    • Stella Graßhof, Hanno Ackermann, Sami Brandt, Jörn Ostermann
      Apathy is the Root of all Expressions
      12th IEEE Conference on Automatic Face and Gesture Recognition (FG2017) (accepted), Washington D.C., USA, 2017
    • Stella Graßhof, Hanno Ackermann, Jörn Ostermann
      Estimation of Face Parameters using Correlation Analysis and a Topology Preserving Prior
      14th IAPR International Conference on Machine Vision Applications (MVA), Tokyo, May 2015