Older Projects (before 1999)

- - - o - - -

3D face database

Using a Cyberware 3D laser scanner, I generated a data base of 3D models of human faces. The scanner records shape and texture of a face simultaneously and with the same resolution, so that each surface coordinate is registered with exactly one texture pixel. The surface points are sampled according to a regular grid in cylinder coordinates. Two neighbouring points on a horizontal scanline are 0.8 degrees apart. Two points on a vertical scanline have a distance of 0.615 mm.

Currently, the main database contains 100 male and 100 female faces of caucasian people aged between 20 and 40 years. The models do not contain hairs and the whole back of the head is removed completely. The faces are aligned in space in order to minimize the sum-squared distances between a selected set of features (such as the pupils, the tip of the nose, and the corners of the mouth).

If the texture is mapped onto the surface model, 2D projections result in images that look like colored photographs taken from the chosen angle. The surface models can also be rendered substituting the realistic texture by homogenous reflection properties and applying an illumination model. The following two animations show an example face rendered without texture and with applying the texture.

Images of the faces rendered from different viewpoints can be downloaded from the Web-page of the MPI for Biological Cybernetics.

- - - o - - -

Recognizing faces from new viewpoints

with Heinrich H. Bülthoff

Although remarkably robust, face recognition is not perfectly invariant to pose and viewpoint changes. It has long been known, that both profile and full-face views result in poorer recognition performance than a 3/4 view. However, little data existed which investigatee this phenomenon in detail. We provide such data using a high angular resolution and a large range of poses. Since there are inconsistencies in the literature concerning these issues, we emphasize the different roles of the learning view and the testing view in the recognition experiment. We also emphazise the roles of information contained in the texture and in the shape of a face. The results of our same/different face recognition experiments are: 1. Only the learning view but not the testing view affects recognition performance. 2. For textured faces the optimal learning view is closer to the full-face view than for the shaded faces. 3. For shaded faces, we find a significantly better recognition performance for the symmetric view. The results can be interpreted in terms of different strategies to recover invariants from texture and from shading.

Publications:

Troje, N. F. and Bülthoff, H. H. (1996) Face recognition under varying pose: The role of texture and shape. Vision Research 36:1761-1771.

- - - o - - -

Bilateral symmetry of faces and its role in face recognition

with Heinrich H. Bülthoff

The role of bilateral symmetry in face recognition is investigated in psychophysical experiments using a same/different paradigm. Our hypothesis is that the ability to identify mirror symmetric images is used for viewpoint generalization by approximating the symmetric view of a learned view by its mirror symmetric image. We could shows that generalization to the symmetric view is better, than generalization to otherwise different views. If the symmetric view is replaced by the mirror reversed learning view, performance further increases. We also showed that the match between the learned view and the test image is performed directly on the level of the images. Performance drops dramatically if the symmetry between the intensity patterns of learning and testing view is disturbed by an asymmetric illumination, although the symmetry between the spatial arrangement of high-level features is retained. We show that a simple image based model can explain important aspects of the data and we show how this model can be extended towards a general algorithm for image comparison.

Publications:

Bilateral symmetry of human faces helps to generalize to novel views. Poster presented at the 24th Göttingen Neurobiology Conference 1996. (PDF)

Troje, N. F. and Bülthoff, H. H. (1998) How is bilateral symmetry of human faces used for recognition of novel views? Vision Research 38:79-89.

Troje, N. F. (1998) Generalization to novel views of faces: Psychophysics and models concerning the role of bilateral symmetry. In: Downward processes in the perception representation mechanisms, C. Taddei-Ferretti and C. Musio (eds), World Scientific, Singapore, New Jersey, London, Hong Kong, pp. 171-188. (PDF)

- - - o - - -

Correspondence-based representations of faces

with Thomas Vetter

Several models for parameterized face representations have been proposed in the last years. A simple coding scheme treats the image of a face as a long vector with each entry coding for the intensity of one single pixel in the image (e.g. Sirovich & Kirby 1987). Although simple and straightforward, such pixel-based representations have several disadvantages. We developed a representation for images of faces that separates texture and 2D shape by exploiting pixel-by-pixel correspondence between the images. The advantages of this representation compared to pixel-based representations are demonstrated by means of the quality of low-dimensional reconstructions derived from principal component analysis and by means of the performance that a simple linear classifier can achieve for sex classification.

Publications:

A separated linear shape and texture space for modeling two-dimensional images of human faces. MPI-Memo No. 15 (PDF)

Pixel-based versus correspondence-based representations of human faces: Implications for sex discrimination. Talk presented at ECVP 96, Strasbourg, September 9 - 13, 1996. (HTML)

Troje, N. F. and Vetter, T. (1998) Representations of human faces. In: Downward processes in the perception representation mechanisms, C. Taddei-Ferretti and C. Musio (eds), World Scientific, Singapore, New Jersey, London, Hong Kong, pp.189-205 (invited). (PDF)

- - - o - - -

Illumination induced apparent shift in head orientation

with Ulrike Siebeck



Changing the position of a light source illuminating a human face induces an apparent shift in perceived orientation (see figure below). The direction of this apparent shift is opposed to the shift of the light source. We investigate this phenomenon in order to achieve some understanding of the mechanisms underlying orientation judgement in general. Using a null method, we measured the illumination induced apparent orientation shift under different experimental conditions. Apparent orientation shifts of more than 10 degrees can be reached. The illumination induced orientation shift disappears if the same faces are shown in front of a lighter background. We conclude that orientation judgement is based on the location of the visible outline in the image and discuss our results in terms of possible mechanisms fororientation judgement.

These two images show the same face in exactly the same orientation. In the left image, the face is illuminated 30 degrees from the right, in the right image, the face is illuminated 30 degrees from the left. The shift in the position of the light source causes an apparent shift in orientation. Have a look at some more demos.

Publications:

Apparent illumination induced orientation shift of human faces depends on the absolute orientation of the face. Poster presented at the 25th Göttingen Neurobiology Conference 1997. (PDF)

Troje, N. F. and Siebeck, U. (1998) Illumination induced apparent shift in orientation of human heads. Perception 27:671-680.