Biological motion contains information about the identity of a moving person. We conducted a series of experiments which aim to illucidate the perceptual aspects of person identification. At the same time, we are developing algorithms for artificial person identification.

Publications:

• Troje, N. F., Westhoff, C., and Lavrov, M. (2005) Person identification from biological motion: Effects of structural and kinematic cues. Perception & Psychophysics 67:667-675. Full Text
• Jokisch, D., Daum, I., and Troje, N.F. (2006). Self recognition versus recognition of others by biological motion: Viewpoint-dependent effects. Perception. (PDF)
• Zang, Z. and Troje, N.F. (2005) View-independent person identification from human gait. Neurocomputing 69:250-256. Full Text

The Functional Significance of Head-Bobbing in Birds

Head movement of a walking pigeon is characterized by two alternating phases — a thrust phase and a hold phase. While the head is rapidly thrust forward during the thrust phase, it remains fixed in space during the hold phase. This is most liely an optokinetic response. However, it remains unclear why some bird species show head-bobbing while others can well survive without it. Furthermore, we don’t yet know whether the thrust phase has a visual function on its own. We are working on both these questions.

Publications:

• Troje, N.F. and Frost, B.J. (1999) Evidence for active vision during the thrust-phase of the pigeon’s head-bobbing. Paper presented at the 9^th Annual Meeting of the Canadian Society for Brain, Behaviour and Cognitive Science, Edmonton, Alberta. Abstract
• Troje, N.F. and Frost, B.J. (2000) Head-bobbing in pigeons: how stable is the hold phase? Journal of Experimental Biology 203:935-940. Full Text / Abstract

The “Virtual Pigeon” project

In collaboration with Barrie Frost we have initiated a program on the neuroethology of conspecific recognition in pigeons. Pigeons do show robust courtship responses not only to real partners but also to video images. View an MPEG video (size 3.2 MB) of a male pigeon courting a female pigeon on the television.

This encouraged us to take this a step further and construct a “virtual pigeon” using high-level computer graphics tools (both  SoftImage and Maya). This gives us complete control about the actions and appearance of the model bird. Several versions have been developed so far.

Virtual Pigeon Evolution Demos

In collaboration with Toru Shimizu we are conducting immunohistochemical studies to investigate which brain areas are involved in social recognition. Furthermore, we are constructing miniature FM telemetry units to transmit electrophysiological data from pigeons as they observe and socially interact with the virtual pigeon.

Publications:

• Frost, B.J., Troje, N.F. and David, S. (1998). Pigeon courtship behaviour in response to live birds and video presentations. 5^th International Congress of Neuroethology. Poster
• Troje, N.F., Frost, B.J. and David, S. (1998) An ethogram of the pigeon’s bowing display. 5^th International Congress of Neuroethology. Poster

   3D Face Database

Using a Cyberware 3D laser scanner, I generated a data base of 3D models of human faces. The scanner records shape and texture of a face simultaneously and with the same resolution, so that each surface coordinate is registered with exactly one texture pixel. The surface points are sampled according to a regular grid in cylinder coordinates. Two neighbouring points on a horizontal scanline are 0.8 degrees apart. Two points on a vertical scanline have a distance of 0.615 mm.

Currently, the main database contains 100 male and 100 female faces of caucasian people aged between 20 and 40 years. The models do not contain hairs and the whole back of the head is removed completely. The faces are aligned in space in order to minimize the sum-squared distances between a selected set of features (such as the pupils, the tip of the nose, and the corners of the mouth).

If the texture is mapped onto the surface model, 2D projections result in images that look like colored photographs taken from the chosen angle. The surface models can also be rendered substituting the realistic texture by homogenous reflection properties and applying an illumination model. The following two animations show an example face rendered without texture and with applying the texture.

shaded face without texture (MPEG 54 K)

textured face (MPEG 48 K)

Images of the faces rendered from different viewpoints can be downloaded from the Web-page of the MPI for Biological Cybernetics.

Recognizing faces from new viewpoints

with Heinrich H. Bülthoff

Although remarkably robust, face recognition is not perfectly invariant to pose and viewpoint changes. It has long been known, that both profile and full-face views result in poorer recognition performance than a 3/4 view. However, little data existed which investigate this phenomenon in detail. We provide such data using a high angular resolution and a large range of poses. Since there are inconsistencies in the literature concerning these issues, we emphasize the different roles of the learning view and the testing view in the recognition experiment. We also emphazise the roles of information contained in the texture and in the shape of a face. The results of our same/different face recognition experiments are: 1. Only the learning view but not the testing view affects recognition performance. 2. For textured faces the optimal learning view is closer to the full-face view than for the shaded faces. 3. For shaded faces, we find a significantly better recognition performance for the symmetric view. The results can be interpreted in terms of different strategies to recover invariants from texture and from shading.

Publications:

• Troje, N. F. and Bülthoff, H. H. (1996) Face recognition under varying pose: The role of texture and shape. Vision Research 36:1761-1771. (PDF)

Bilateral symmetry of faces and its role in face recognition

with Heinrich H. Bülthoff

The role of bilateral symmetry in face recognition is investigated in psychophysical experiments using a same/different paradigm. Our hypothesis is that the ability to identify mirror symmetric images is used for viewpoint generalization by approximating the symmetric view of a learned view by its mirror symmetric image. We could shows that generalization to the symmetric view is better, than generalization to otherwise different views. If the symmetric view is replaced by the mirror reversed learning view, performance further increases. We also showed that the match between the learned view and the test image is performed directly on the level of the images. Performance drops dramatically if the symmetry between the intensity patterns of learning and testing view is disturbed by an asymmetric illumination, although the symmetry between the spatial arrangement of high-level features is retained. We show that a simple image based model can explain important aspects of the data and we show how this model can be extended towards a general algorithm for image comparison.

Publications:

• Bilateral symmetry of human faces helps to generalize to novel views. Poster presented at the 24th Göttingen Neurobiology Conference 1996. (PDF)
• Troje, N. F. and Bülthoff, H. H. (1998) How is bilateral symmetry of human faces used for recognition of novel views? Vision Research 38:79-89. (PDF)
• Troje, N. F. (1998) Generalization to novel views of faces: Psychophysics and models concerning the role of bilateral symmetry. In: Downward processes in the perception representation mechanisms, C. Taddei-Ferretti and C. Musio (eds), World Scientific, Singapore, New Jersey, London, Hong Kong, pp. 171-188. (PDF)

Correspondence-based representations of faces

with Thomas Vetter

Several models for parameterized face representations have been proposed in the last years. A simple coding scheme treats the image of a face as a long vector with each entry coding for the intensity of one single pixel in the image (e.g. Sirovich & Kirby 1987). Although simple and straightforward, such pixel-based representations have several disadvantages. We developed a representation for images of faces that separates texture and 2D shape by exploiting pixel-by-pixel correspondence between the images. The advantages of this representation compared to pixel-based representations are demonstrated by means of the quality of low-dimensional reconstructions derived from principal component analysis and by means of the performance that a simple linear classifier can achieve for sex classification.

Publications:

• A separated linear shape and texture space for modeling two-dimensional images of human faces. MPI-Memo No. 15 (PDF)
• Pixel-based versus correspondence-based representations of human faces: Implications for sex discrimination. Talk presented at ECVP 96, Strasbourg, September 9 – 13, 1996. (HTML)
• Troje, N. F. and Vetter, T. (1998) Representations of human faces.In: Downward processes in the perception representation mechanisms, C. Taddei-Ferretti and C. Musio (eds), World Scientific, Singapore, New Jersey, London, Hong Kong, pp.189-205 (invited). (PDF)

Illumination induced apparent shift in head orientation

with Ulrike Siebeck

Changing the position of a light source illuminating a human face induces an apparent shift in perceived orientation (see figure below). The direction of this apparent shift is opposed to the shift of the light source. We investigate this phenomenon in order to achieve some understanding of the mechanisms underlying orientation judgement in general. Using a null method, we measured the illumination induced apparent orientation shift under different experimental conditions. Apparent orientation shifts of more than 10 degrees can be reached. The illumination induced orientation shift disappears if the same faces are shown in front of a lighter background. We conclude that orientation judgement is based on the location of the visible outline in the image and discuss our results in terms of possible mechanisms fororientation judgement.

These two images show the same face in exactly the same orientation. In the left image, the face is illuminated 30 degrees from the right, in the right image, the face is illuminated 30 degrees from the left. The shift in the position of the light source causes an apparent shift in orientation. Have a look at some more demos.

Publications:

• Apparent illumination induced orientation shift of human faces depends on the absolute orientation of the face. Poster presented at the 25th Göttingen Neurobiology Conference 1997.(PDF)
• Troje, N. F. and Siebeck, U. (1998) Illumination induced apparent shift in orientation of human heads. Perception 27:671-680. (PDF)