A Tracking Framework for Accurate Face Localization

Achieving a good localization of faces on video frames is of high importance for an application such as video indexing. Face localization on movies is an ambiguous task due to various scale, pose and lighting conditions.


Our Method

A novel deterministic approach has been developed that it applies face detection, forward tracking and backward tracking, using some predefined rules. From all the possible extracted candidates, a Dynamic Programming algorithm selects those that minimize a cost function.

Face detection:

Use of the Haar-like features to provide first face candidates. For tracking purposes, a post-processing step is added to reduce the number of false alarms and remove some of the background. Candidates are rejected if the number of pixels fulfilling the criteria below are under a certain threshold.

0 < h < 1 and 0.23 < s < 0.68 and 0.27 < v

The remaining candidates are replaced by the smallest bounding box containing the skin-like pixels. The detection is performed every 5 frames.

Forward tracking process:

  • The Morphological Elastic Graph Matching (MEGM) tracking algorithm is used.
  • The tracking is initialized by the detection output.
  • While the detection is efficient for frontal faces, the tracking provides candidates for other possible poses.

Backward tracking process:

  • If a face is detected in the middle of a shot, no information is available about its localization in the previous video frames.
  • The same tracker (MEGM) is applied backwards.
  • The backward tracker is initialized with the labeled detections.
  • Detected, forward and backward tracked faces are now grouped in actor appearances (labels).


  • The node cost C is the distance between the center of the bounding box and the centroid of the skin-like pixels.
  • The transition cost combines:
    • the overlap between two bounding boxes
    • the ratio of the bounding boxes areas to penalize big changes in the area during tracking

Structure of the trellis

  • The trellis selects the path with the minimum cost. The selection is performed using dynamic programming.

Experimental results





Relevant Publications

I. Cherif, V. Solachidis and I. Pitas, "A Tracking Framework for Accurate Face Localization", in Proc. of Int. Federation for Information Processing Conf. on Artificial Intelligence (IFIP AI 2006), Santiago, Chile, 21-24 August, 2006.


Research Projects

NM2 - “New media for a new millennium” (IST-004124), FP6


© 2006