This chapter starts with a general and brief introduction of biometrics and audiovisual person recognition using mobile phone data. It begins with a discussion of what constitutes a biometric recognition system, and it then details the steps followed when audio-visual signals are used as inputs. This is followed by a review of the existing speaker and face recognition systems which have been evaluated on a mobile biometric database. We then discuss the key motivations of using deep neural network (DNN) for person recognition. We finally introduce a Deep Boltzmann Machine (DBM)- DNN, in short DBM-DNN, based framework for person recognition. An overview of the sections and sub-sections of this chapter is shown in Figure 4.1.

Chapter Contents:

• 4.1 Biometric systems
• 4.1.1 What is biometrics?
• 4.1.2 Multimodal biometrics
• 4.2 Audio-visual biometric systems
• 4.2.1 Preprocessing
• 4.2.2 Feature extraction
• 4.2.2.1 Acoustic features
• 4.2.2.2 Visual features
• 4.2.3 Classification
• 4.2.3.1 Support vector machine (SVM)
• 4.2.3.2 Linear regression-based classifier (LRC)
• 4.2.4 Fusion
• 4.2.4.1 Non-adaptive fusion
• 4.2.4.2 Adaptive fusion
• 4.2.4.3 Linear logistic regression (logReg) fusion
• 4.2.5 Audio-visual corporation
• 4.2.5.1 BANCA
• 4.2.5.2 M2VTS and XM2VTS
• 4.2.5.3 VidTIMIT
• 4.2.5.4 AusTalk
• 4.2.5.5 MOBIO
• 4.3 Mobile person recognition
• 4.3.1 Speaker recognition systems
• 4.3.1.1 GMM-based methods
• 4.3.1.2 Parts-based method
• 4.3.2 Face recognition systems
• 4.3.2.1 Template-based
• 4.3.2.2 GMM based
• 4.3.3 Audio-visual person recognition on MOBIO
• 4.4 Deep neural networks for person recognition
• 4.4.1 A DBN-DNN for unimodal person recognition
• 4.4.2 A DBM-DNN for person recognition
• 4.4.2.1 DBM training
• 4.4.2.2 Person recognition using DBM-DNN
• 4.5 Summary
• References

Inspec keywords: speaker recognition; face recognition; mobile computing; audio signal processing; database management systems; biometrics (access control); Boltzmann machines

Other keywords: deep neural networks; mobile phone data; speaker recognition; audio-visual signals; face recognition; DBM; biometrics; DNN; audiovisual person recognition; mobile person; mobile biometric database; deep Boltzmann machine

Subjects: Mobile, ubiquitous and pervasive computing; Image recognition; Database management systems (DBMS); Speech processing techniques; Data security; Computer vision and image processing techniques; Speech recognition and synthesis; Neural computing techniques