Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free HPILN: a feature learning framework for cross-modality person re-identification

© The Institution of Engineering and Technology
Received 06/06/2019, Accepted 16/09/2019, Revised 19/08/2019, Published 19/09/2019

Most video surveillance systems use both RGB and infrared cameras, making it a vital technique to re-identify a person cross the RGB and infrared modalities. This task can be challenging due to both the cross-modality variations caused by heterogeneous images in RGB and infrared, and the intra-modality variations caused by the heterogeneous human poses, camera position, light brightness etc. To meet these challenges, a novel feature learning framework, hard pentaplet and identity loss network (HPILN), is proposed. In the framework existing single-modality re-identification models are modified to fit for the cross-modality scenario, following which specifically designed hard pentaplet loss and identity loss are used to increase the accuracy of the modified cross-modality re-identification models. Based on the benchmark of the SYSU-MM01 dataset, extensive experiments have been conducted, showing that the authors’ method outperforms all existing ones in terms of cumulative match characteristic curve and mean average precision.

Inspec keywords: image classification; video surveillance; cameras; feature extraction; pose estimation; image colour analysis; learning (artificial intelligence)

Other keywords: heterogeneous human poses; cameras; modified cross-modality re-identification models; video surveillance systems; mean average precision; cross-modality person; SYSU-MM01 dataset; single-modality re-identification models; HPILN; feature learning framework; heterogeneous images; intra-modality variations; cross-modality variations; cross-modality scenario; identity loss network; cumulative match characteristic curve; RGB; camera position

Subjects: Computer vision and image processing techniques; Image recognition; Video signal processing; Knowledge engineering techniques

References

    1. 1)
      • 2. Yu, Q., Chang, X., Song, Y.Z., et al: ‘The devil is in the middle: exploiting mid-level representations for cross-domain instance matching’. arXiv preprint arXiv:1711.08106, 2017.
    2. 2)
      • 6. Wang, G., Yuan, Y., Chen, X., et al: ‘Learning discriminative features with multiple granularities for person re-identification’. ACM Int. Conf. on Multimedia, Seoul, South Korea, 2018, pp. 274–282.
    3. 3)
      • 39. Wen, Y., Zhang, K., Li, Z., et al: ‘A discriminative feature learning approach for deep face recognition’. Proc. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 499–515.
    4. 4)
      • 19. Qian, X., Fu, Y., Jiang, Y.G., et al: ‘Multi-scale deep learning architectures for person re-identification’. Proc. 2017 IEEE Int. Conf. on Computer Vision (ICCV), Venice, Italy, 2017, pp. 5409–5418.
    5. 5)
      • 32. Chen, Y.C., Zhu, X., Zheng, W.S., et al: ‘Person re-identification by camera correlation aware feature augmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2018, 40, (2), pp. 392–408.
    6. 6)
      • 24. Kniaz, V.V., Knyaz, V.A., Hladuvka, J., et al: ‘Thermalgan: multimodal color-to-thermal image translation for person re-identification in multispectral dataset’. Proc. European Conf. on Computer Vision (ECCV), Munich, Germany, 2018.
    7. 7)
      • 23. Chattopadhyay, P., Sural, S., Mukherjee, J.: ‘Information fusion from multiple cameras for gait-based re-identification and recognition’, IET Image Process., 2015, 9, (11), pp. 969–976.
    8. 8)
      • 14. Schroff, F., Kalenichenko, D., Philbin, J.: ‘FaceNet: a unified embedding for face recognition and clustering’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Boston, USA, 2015, pp. 815–823.
    9. 9)
      • 12. Farenzena, M., Bazzani, L., Perina, A., et al: ‘Person re-identification by symmetry-driven accumulation of local features’. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, San Fransisco, USA, 2010, pp. 2360–2367.
    10. 10)
      • 11. Wang, Z., Wang, Z., Zheng, Y., et al: ‘Learning to reduce dual-level discrepancy for infrared-visible person re-identification’. Proc. IEEE Int. Conf. on Artificial Intelligence Organisation/IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, 2019, pp. 618–626.
    11. 11)
      • 15. Hermans, A., Beyer, L., Leibe, B.: ‘In defense of the triplet loss for person re-identification’. arXiv preprint arXiv:1703.07737, 2017.
    12. 12)
      • 13. Varior, R.R., Haloi, M., Wang, G.: ‘Gated Siamese convolutional neural network architecture for human re-identification’. Proc. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 791–808.
    13. 13)
      • 38. Yoon, K., Song, Y.M., Jeon, M.: ‘Multiple hypothesis tracking algorithm for multi-target multi-camera tracking with disjoint views’, IET Image Process., 2018, 12, (7), pp. 1175–1184.
    14. 14)
      • 20. Li, W., Zhu, X., Gong, S.: ‘Harmonious attention network for person re-identification’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 2285–2294.
    15. 15)
      • 28. Hao, Y., Wang, N., Li, J., et al: ‘HSME: hypersphere manifold embedding for visible thermal person re-identification’. Proc. AAAI Conf. on Artificial Intelligence, Honolulu, Hawaii, USA, 2019, vol. 33, pp. 8385–8392.
    16. 16)
      • 25. Moon, H., Phillips, P.J.: ‘Computational and performance aspects of PCA-based face-recognition algorithms’, Perception, 2001, 30, (3), pp. 303–321.
    17. 17)
      • 26. Kingma, D.P., Ba, J.: ‘Adam: a method for stochastic optimization’. arXiv preprint arXiv:1412.6980, 2014.
    18. 18)
      • 41. Huang, G., Liu, Z., Van Der Maaten, L., et al: ‘Densely connected convolutional networks’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 4700–4708.
    19. 19)
      • 17. Xiao, T., Li, H., Ouyang, W., et al: ‘Learning deep feature representations with domain guided dropout for person re-identification’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 1249–1258.
    20. 20)
      • 37. Ristani, E., Solera, F., Zou, R., et al: ‘Performance measures and a data set for multi-target, multi-camera tracking’. Proc. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 17–35.
    21. 21)
      • 40. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 770–778.
    22. 22)
      • 27. Ye, M., Lan, X., Wang, Z., et al: ‘Bi-directional center-constrained top-ranking for visible thermal person re-identification’, IEEE Trans. Inf. Forensics Sec., 2020, 15, pp. 407–419.
    23. 23)
      • 4. Chang, X., Hospedales, T.M., Xiang, T.: ‘Multi-level factorisation net for person re-identification’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 2109–2118.
    24. 24)
      • 36. Li, W., Zhao, R., Xiao, T., et al: ‘DeepReID: deep filter pairing neural network for person re-identification’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 152–159.
    25. 25)
      • 3. Sun, Y., Zheng, L., Yang, Y., et al: ‘Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline)’. European Conf. on Computer Vision (ECCV), Munich, Germany, 2018, pp. 480–496.
    26. 26)
      • 33. Rasiwasia, N., Costa Pereira, J., Coviello, E., et al: ‘A new approach to cross-modal multimedia retrieval’. Proc. 18th ACM Int. Conf. on Multimedia, Firenze, Italy, 2010, pp. 251–260.
    27. 27)
      • 21. Wu, A., Zheng, W.S., Lai, J.H.: ‘Robust depth-based person re-identification’, IEEE Trans. Image Process., 2017, 26, (6), pp. 2588–2603.
    28. 28)
      • 18. Chen, H., Wang, Y., Shi, Y., et al: ‘Deep transfer learning for person re-identification’. Proc. 2018 IEEE Fourth Int. Conf. on Multimedia Big Data (BigMM), Xi'an, China, 2018, pp. 1–5.
    29. 29)
      • 29. Dalal, N., Triggs, B.: ‘Histograms of oriented gradients for human detection’. Proc. IEEE Int. Conf. on Computer Vision & Pattern Recognition (CVPR'05), San Diego, CA, USA, 2005, vol. 1, pp. 886–893.
    30. 30)
      • 34. Pedagadi, S., Orwell, J., Velastin, S., et al: ‘Local fisher discriminant analysis for pedestrian re-identification’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013, pp. 3318–3325.
    31. 31)
      • 16. Chen, W., Chen, X., Zhang, J., et al: ‘Beyond triplet loss: a deep quadruplet network for person re-identification’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 403–412.
    32. 32)
      • 1. Zheng, L., Yang, Y., Hauptmann, A.G.: ‘Person re-identification: past, present and future’. arXiv preprint arXiv:1610.02984, 2016.
    33. 33)
      • 5. Dai, Z., Chen, M., Zhu, S., et al: ‘Batch feature erasing for person re-identification and beyond’. arXiv preprint arXiv:1811.07130, 2018.
    34. 34)
      • 10. Kang, J.K., Hoang, T.M., Park, K.R., et al: ‘Person re-identification between visible and thermal camera images based on deep residual CNN using single input’, IEEE Access, 2019, 7, pp. 57972–57984.
    35. 35)
      • 8. Ye, M., Wang, Z., Lan, X., et al: ‘Visible thermal person re-identification via dual-constrained top-ranking’. Proc. Int. Joint Conf. on Artificial Intelligence, Stockholm, Sweden, 2018, pp. 1092–1099.
    36. 36)
      • 22. Barbosa, I.B., Cristani, M., Del Bue, A., et al: ‘Re-identification with RGB-D sensors’. Proc. European Conf. on Computer Vision, Florence, Italy, 2012, pp. 433–442.
    37. 37)
      • 9. Dai, P., Ji, R., Wang, H., et al: ‘Cross-modality person re-identification with generative adversarial training’. Proc. Int. Joint Conf. on Artificial Intelligence, Stockholm, Sweden, 2018, pp. 677–683.
    38. 38)
      • 42. Selvaraju, R.R., Cogswell, M., Das, A., et al: ‘Grad-CAM: visual explanations from deep networks via gradient-based localization’. Proc. IEEE Int. Conf. on Computer Vision, Venice, Italy, 2017, pp. 618–626.
    39. 39)
      • 7. Wu, A., Zheng, W.S., Yu, H.X., et al: ‘RGB-infrared cross-modality person re-identification’. Proc. IEEE Int. Conf. on Computer Vision, Venice, Italy, 2017, pp. 5380–5389.
    40. 40)
      • 30. Liao, S., Hu, Y., Zhu, X., et al: ‘Person re-identification by local maximal occurrence representation and metric learning’. Proc. IEEE Int. Computer Society Conf. on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, pp. 2197–2206.
    41. 41)
      • 31. Lin, D., Tang, X.: ‘Inter-modality face recognition’. Proc. European Conf. on Computer Vision, Graz, Austria, 2006, pp. 13–26.
    42. 42)
      • 35. Zheng, L., Shen, L., Tian, L., et al: ‘Scalable person re-identification: a benchmark’. Proc. IEEE Int. Conf. on Computer Vision, Santiago, Chile, 2015, pp. 1116–1124.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2019.0699
Loading

Related content

content/journals/10.1049/iet-ipr.2019.0699
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address