In this video, we will give an overview of what
our facial attributes give a little bit of background on
their detection and show how
a deep convolutional neural network can be applied to the problem of their recognition.
Facial attributes such as gender, race, age,
or presence of facial hair such as
a mustache arise when examining performance of face recognition algorithms,
particularly those that aim at searching for specific people in large databases.
A common case in video surveillance applications.
These not only might the facial pose differ,
but so might for instance,
the expression and hairstyle.
As a result, in these applications,
men might get confused for women,
young people for old,
or Asians for Caucasians.
All these are examples of facial attributes.
High level visual traits describing an appearance of a face image.
In general, attributes are considered insensitive to pose,
illumination expression and other imaging conditions.
One more application of visual attributes might be image search.
Suppose one is to find images with
specified facial appearance such as the smiling Asian man with glasses.
When using annotation-based image search,
one confronts with sparsely annotated data that might be time
consuming to label and prone to incorrect or misleading annotations.
Image retrieval then might yield a plethora of
irrelevant or incomplete results indicating
that annotation-based image search does not scale.
Alternatively, attribute-based image retrieval system recognizes
attributes and automatically tags images with attribute labels to make them searchable.
Facial attribute detection pipeline typically
involves localizing face or facial landmarks in a high resolution image.
This tap is performed using
an object detection or a key point progression convolutional neural network,
that we will discuss further in the course.
The detected face is normalized and passed to
an attribute classification convolutional neural net
to predict one or more face attributes.
Facial attributes recognition is typically evaluated using
a simple prediction accuracy metric
corresponding to a fraction of correctly predicted facial attributes.
Attribute recognition methods are generally subdivided into two broad groups,
global and local methods.
Global methods extract features from the entire object,
while accurate locations of object parts or landmarks are not required.
They are not robust to defamations of objects.
Recently, local methods were proposed that first
detect object parts and extract features from each part.
These local features are then concatenated to train classifiers.
Let us describe both approaches in greater detail.
Gathering a large labeled image training set for such attribute as age and gender
from social image repositories requires
either access to personal information on the subjects appearing in the images,
which is often private,
or the tedious and time consuming manual labeling.
Data-sets for age and gender estimation from
real world social images are therefore relatively limited in size.
And presently, no match in size with
a much larger image classification data-sets such as the ImageNet.
Overfitting is common problem when
machine learning based methods are used on such small image collections.
This problem is exacerbated when considering
deep convolutional neural networks due to
their huge number of trainable model parameters.
Care must be therefore taken in order to avoid overfitting under such circumstances.
Ways to avoid overfitting include employing aggressive data augmentation strategies,
such as random cropping and mirroring input data.
And usually, people use multiple normalization
and drop-out layers and inject them in the network to stabilize learning.
For certain attribute types,
age group classification rather than precise age estimation may improve performance
because this implies learning the cross-entropy loss and not the usual L2 loss.
One may adopt a multistage attribute recognition
procedure and train separate convolutional neural network,
face detector and attribute classified models.
While the detector is trained in a relatively straightforward way,
and we'll get to this during the next week,
attribute classifier is pre-trained on
a large collection of images for face verification.
By pre-training on face verification task,
attribute classifier learns to extract face sensitive convolutional features.
It is later fine tuned to adapt to the final task of attribute recognition.
Research shows pre-training on
the face verification task significantly improves classifier performance.
During inference, accuracy is further improved via computing on multiple crops.
To summarize, attributes serve as describable traits of visual appearance,
and they are important for face recognition and face retrieval.
There are local and global attribute recognition approaches,
and they differ in how they perform
feature extraction from facial images and interpret these results.
As usual, employing convolutional neural network
for facial image classification requires careful tuning of network hyperparameters,
pre-training and data augmentation.
Anton KonushinSenior Lecturer
Alexey ArtemovSenior Lecturer
