Face Clustering

Face clustering is a very important application for movie's semantic extraction. It can contribute in many ways, like determining the primordial actors or the creation of databases references or dialog detection and many others.

Mutual information (MI) is a novel and useful tool in order to find similarities between information. More concretely, MI is defined as the information that is shared between two distributions.


Our Method

Our approach exploits the capabilities of joint entropy and mutual information in order to classify face images exported from a Haar detector. We use the intensity images and we define for every image the probability density function as the histogram of the intensities of that image summed to one. In order to calculate the joint entropy between the two images we construct a 2D histogram of 256 bins which take in account the relative positions of intensities so that similarity occurs between two images, when same intensities are located in same spatial locations.

Problem Definition

  • Face clustering is the task where from a set of face images A we create n subsets .
  • We exploit the mutual information between the face images to create a similarity matrix which afterwards will be clustered.
  • The mutual information is shown to be a good measure for similarity between face images where light conditions and poses are variant.
  • The movies' context for such an algorithm gives a new dimension to the problem where no calibrated images are used as input.
  • Purpose of such an algorithm: Define primordial actors, automatic (not manually annotated) data base search, registration, content analysis.

Clustering Process:

  • The clustering process is based on the Fuzzy-C Means (FCM) algorithm.
  • We provide the number of classes and the similarity matrix to the algorithm.
  • In order to use this algorithm we define every row of the aforementioned similarity matrix as a different vector in an M-dimensional L2-normed vector space over R.
Darker regions belong to the first actor and clearer ones to the second actor. The video sequence has four consecutive shots in the order FA-FA-SA-SA where FA and SA first and second actor respectively





Relevant Publications

Ν. Vretos, V. Solachidis and I. Pitas "A Mutual Information Based Algorithm for Face Clustering", in Proc. of Int. Conf. on Multimedia and Expo (ICME 2006) , Toronto Ontario, Canada, 9-12 July, 2006.


Research Projects

NM2 - “New media for a new millennium” (IST-004124), FP6


© 2006