© The Institution of Engineering and Technology
Automated driving will have a big impact on society, creating new possibilities for mobility and reducing road accidents. Current developments aim to provide driver assistance in the form of conditional and partial automation. Computer vision, either alone or combined with other technologies such as radar or lidar, is one of the key technologies of advanced driver assistance systems (ADAS). The presence of vision technologies inside the vehicles is expected to grow as the automation levels increase. However, embedding a vision-based driver assistance system supposes a big challenge due to the special features of vision algorithms, the existing constrains and the strict requirements that need to be fulfilled. The aim of this study is to show the current progress and future directions in the field of vision-based embedded ADAS, bridging the gap between theory and practice. The different hardware and software options are reviewed, and design, development and testing considerations are discussed. Additionally, some outstanding challenges are also identified.
References
-
-
1)
-
36. Schwiegelshohn, F., Gierke, L., Michael, H.: ‘FPGA based traffic sign detection for automotive camera systems’. 2015 10th Int. Symp. on Reconfigurable Communication-Centric Systems-on-Chip (ReCoSoC), 2015, pp. 1–6.
-
2)
-
38. Son, S., Baek, Y.: ‘Design and implementation of real-time vehicular camera for driver assistance and traffic congestion estimation’, Sensors, 2015, 15, (8), pp. 20204–20231.
-
3)
-
8. Forster, F.: ‘Heterogeneous processors for advanced driver assistance systems’, ATZelektronik Worldw., 2014, 9, (1), pp. 14–18.
-
4)
-
46. Manoharan, R., Chandrakala, S.: ‘Android OpenCV based effective driver fatigue and distraction monitoring system’. 2015 Int. Conf. on Computing and Communications Technologies (ICCCT'15), 2015, pp. 262–266.
-
5)
-
47. Honegger, D., Oleynikova, H., Pollefeys, M.: ‘Real-time and low latency embedded computer vision hardware based on a combination of FPGA and mobile CPU’. 2014 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS 2014), 2014, pp. 4930–4935.
-
6)
-
7)
-
50. Nikolic, Z.: ‘Embedded vision in advanced driver assistance systems’, in Kisačanin, B., Gelautz, M. (Eds.): ‘Advances in embedded computer vision’ (Springer International Publishing, 2014), pp. 45–69.
-
8)
-
20. Jheng, Y., Yen, Y., Sun, T.: ‘A symmetry-based forward vehicle detection and collision warning system on android smartphone’. 2015 IEEE Int. Conf. on Consumer Electronics – Taiwan (ICCE-TW), 2015, pp. 212–213.
-
9)
-
18. Nieto, M., Vélez, G., Otaegui, O., et al: ‘Optimising computer vision based ADAS: vehicle detection case study’, IET Intell. Transp. Syst., 2016, 10, (3), pp. 157–164.
-
10)
-
75. ‘Autonomous vehicles need experimental ethics: are we ready for utilitarian cars?’, , 2015.
-
11)
-
17. Chen, G., Shen, P., Cho, C., et al: ‘A forward collision avoidance system adopting multi-feature vehicle detection’. 2014 IEEE Int. Conf. on Consumer Electronics – Taiwan (ICCE-TW), 2014, pp. 125–126.
-
12)
-
55. Elektrobit: ‘EB Assist ADTF’. .
-
13)
-
56. Intempora: ‘RTMaps’. .
-
14)
-
44. Zhang, B., Appia, V., Pekkucuksen, I., et al: ‘A surround view camera solution for embedded systems’. IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), 2014, pp. 662–667.
-
15)
-
4. Aviz, A., Laprie, J., Randell, B.: ‘Fundamental concepts of dependability’. LAAS-CNRS, Technical Report N01145, 2001.
-
16)
-
53. Stein, F.: ‘The challenge of putting vision algorithms into a car’. 2012 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, 2012, pp. 89–94.
-
17)
-
68. Teich, J.: ‘Hardware/software codesign : the past, the present, and predicting the future’, Proc. IEEE, 2012, 100, , pp. 1411–1430.
-
18)
-
1. Bengler, K., Dietmayer, K., Färber, B., et al: ‘Three decades of driver assistance systems: review and future perspectives’, IEEE Intell. Transp. Syst. Mag., 2014, 6, (4), pp. 6–22.
-
19)
-
59. Vicomtech-IK4: ‘Viulib's video annotator’. .
-
20)
-
74. Petit, J., Shladover, S.E.: ‘Potential cyberattacks on automated vehicles’, IEEE Trans. Intell. Transp. Syst., 2015, 16, (2), pp. 546–556.
-
21)
-
34. Yin, S., Ouyang, P., Liu, L., et al: ‘Fast traffic sign recognition with a rotation invariant binary pattern based feature’, Sensors, 2015, 15, (1), pp. 2161–2180.
-
22)
-
5. Kim, K., Choi, K.: ‘SoC architecture for automobile vision system’, in Kim, J., Shin, H. (Eds.): ‘Algorithm & SoC design for automotive vision systems’ (Springer, Netherlands, 2014), pp. 163–195.
-
23)
-
66. Gruyer, D., Choi, S., Boussard, C., et al: ‘From virtual to reality, how to prototype, test and evaluate new ADAS: application to automatic car parking’. 2014 IEEE Intelligent Vehicles Symp., 2014, pp. 261–267.
-
24)
-
35. Borrmann, J.M., Haxel, F., Viehl, A., et al: ‘Safe and efficient runtime resource management in heterogeneous systems for automated driving’. 2015 IEEE 18th Int. Conf. on Intelligent Transportation Systems, 2015, pp. 353–360.
-
25)
-
21. Gu, Q., Yang, J., Zhai, Y., et al: ‘Vision-based multi-scaled vehicle detection and distance relevant mix tracking for driver assistance system’, Opt. Rev., 2015, 22, (2), pp. 197–209.
-
26)
-
52. Koopman, P., Wagner, M.: ‘Challenges in autonomous vehicle testing and validation’, SAE Int. J. Transp. Saf., 2016, 4, (1), pp. 15–24.
-
27)
-
9. Buehler, M., Iagnemma, K., Singh, S. (Eds.): ‘The DARPA urban challenge: autonomous vehicles in city traffic’ (Springer, 2009).
-
28)
-
48. Ahrens, D., Frey, A., Pfeiffer, A., et al: ‘Objective evaluation of software architectures in driver assistance systems’, Comput. Sci. – Res. Dev., 2013, 28, pp. 23–43.
-
29)
-
30)
-
54. Birch, J., Rivett, R., Habli, I., et al: ‘Safety cases and their role in ISO 26262 functional safety assessment’, in Bitsch, F., Guiochet, J., Kaâniche, M. (Eds.): ‘Computer safety, reliability, and security’ (Springer, Berlin Heidelberg, 2013), pp. 154–165.
-
31)
-
10. Darouich, M., Guyetant, S., Lavenier, D.: ‘A reconfigurable disparity engine for stereovision in advanced driver assistance systems’, Lect. Notes Comput. Sci., 2010, 5992, pp. 306–317.
-
32)
-
7. Mukhtar, A., Xia, L., Tang, T.B.: ‘Vehicle detection techniques for collision avoidance systems: a review’, IEEE Trans. Intell. Transp. Syst., 2015, 16, (5), pp. 2318–2338.
-
33)
-
43. Scharfenberger, C., Chakraborty, S., Färber, G.: ‘Robust image processing for an omnidirectional camera-based smart car door’, ACM Trans. Embed. Comput. Syst., 2013, 11, (4), pp. 87:1–87:28.
-
34)
-
42. Almagambetov, A., Velipasalar, S., Member, S., et al: ‘Robust and computationally lightweight autonomous tracking of vehicle taillights and signal detection by embedded smart cameras’, IEEE Trans. Ind. Electron., 2015, 62, (6), pp. 3732–3741.
-
35)
-
60. ‘IEEE Guide – Adoption of the Project Management Institute (PMI) Standard A Guide to the Project Management Body of Knowledge (PMBOK GuiDE) – Fourth Edition’, 2011.
-
36)
-
64. Vanholme, B., Gruyer, D., Glaser, S., et al: ‘Fast prototyping of a highly autonomous cooperative driving system for public roads’. 2010 IEEE Intelligent Vehicles Symp., 2010, pp. 135–142.
-
37)
-
15. Lin, H.-Y., Chen, L.-Q., Lin, Y.-H., et al: ‘Lane departure and front collision warning using a single camera’. 2012 IEEE Int. Symp. on Intelligent Signal Processing and Communications Systems, 2012, pp. 64–69.
-
38)
-
73. Saussard, R., Bouzid, B., Vasiliu, M., et al: ‘Optimal performance prediction of ADAS algorithms on embedded parallel architectures’. 2015 IEEE 17th Int. Conf. on High Performance Computing and Communications (HPCC), 2015, pp. 213–218.
-
39)
-
29. Satzoda, R.K., Lee, S., Lu, F., et al: ‘Snap-DAS: a vision-based driver assistance system on a snapdragon TM embedded platform’. IEEE Intelligent Vehicles Symp., 2015, pp. 660–665.
-
40)
-
28. Velez, G., Cortés, A., Nieto, M., et al: ‘A reconfigurable embedded vision system for advanced driver assistance’, J. Real-Time Image Process., 2015, 10, (4), pp. 725–739.
-
41)
-
23. Petrovai, A., Danescu, R., Nedevschi, S.: ‘A stereovision based approach for detecting and tracking lane and forward obstacles on mobile devices’. 2015 IEEE Intelligent Vehicles Symp., 2015, pp. 634–641.
-
42)
-
26. Anders, J., Mefenza, M., Bobda, C., et al: ‘A hardware/software prototyping system for driving assistance investigations’, J. Real-Time Image Process., 2013.
-
43)
-
61. Geiger, A., Lenz, P., Urtasun, R.: ‘Are we ready for autonomous driving? The KITTI vision benchmark suite’. Conf. on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 3354–3361.
-
44)
-
49. AUTOSAR: ‘AUTOSAR specifications’. .
-
45)
-
14. NVIDIA: ‘NVIDIA DRIVE PX’. .
-
46)
-
24. Gruyer, D., Livic, I.C., Lusetti, B., et al: ‘PerSEE : a central sensors fusion electronic control unit for the development of perception-based ADAS’. 14th IAPR Int. Conf. on Machine Vision Applications (MVA), 2015, pp. 250–254.
-
47)
-
69. SAE J3016_201401: ‘Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems’, 2014.
-
48)
-
49)
-
57. MISRA: ‘MISRA Homepage’. .
-
50)
-
27. Chiang, H., Chen, Y., Member, S., et al: ‘Embedded driver-assistance system using multiple sensors for safe overtaking maneuver’, IEEE Syst. J., 2014, 8, (3), pp. 681–698.
-
51)
-
70. Henzinger, T.A.: ‘Two challenges in embedded systems design: predictability and robustness’, Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci., 2008, 366, (1881), pp. 3727–3736.
-
52)
-
72. Saussard, R., Bouzid, B., Vasiliu, M., et al: ‘The embeddability of lane detection algorithms on heterogeneous architectures’. IEEE Int. Conf. on Image Processing (ICIP), 2015, pp. 4694–4697.
-
53)
-
16. Wu, B., Chen, Y., Yeh, C., et al: ‘Reasoning-based framework for driving safety monitoring using driving event recognition’, IEEE Trans. Intell. Transp. Syst., 2013, 14, (3), pp. 1231–1241.
-
54)
-
41. Chen, Y.-L., Chiang, H.-H., Chiang, C.-Y., et al: ‘A vision-based driver nighttime assistance and surveillance system based on intelligent image sensing techniques and a heterogamous dual-core embedded system architecture’, Sensors, 2012, 12, (12), pp. 2373–2399.
-
55)
-
37. Chiang, C., Chen, Y., Ke, K., et al: ‘Real-time pedestrian detection technique for embedded driver assistance systems’. 2015 IEEE Int. Conf. on Consumer Electronics (ICCE), 2015, pp. 206–207.
-
56)
-
12. Stein, G.P., Rushinek, E., Hayun, G., et al: ‘A computer vision system on a chip: a case study from the automotive domain’. 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR'05) – Workshops, 2005, pp. 130–130.
-
57)
-
3. Laprie, J.: ‘Dependable computing: concepts, limits, challenges’. 25th Int. Symp. on Fault-Tolerant Computing, 1995, pp. 42–54.
-
58)
-
58. Vondrick, C., Ramanan, D., Patterson, D.: ‘Efficiently scaling up video annotation with crowdsourced marketplaces’, in Daniilidis, K., Maragos, P., Paragios, N. (Eds.): ‘Computer vision – ECCV 2010’ (Springer, Berlin Heidelberg, 2010), pp. 610–623.
-
59)
-
45. Neumann, L., Vanholme, B., Gressmann, M., et al: ‘Free space detection: a corner stone of automated driving’. 2015 IEEE 18th Int. Conf. on Intelligent Transportation Systems, 2015, pp. 1280–1285.
-
60)
-
2. World Health Organization: ‘Global status report on road safety 2015’, 2015.
-
61)
-
25. An, X., Shang, E., Song, J., et al: ‘Real-time lane departure warning system based on a single FPGA’, EURASIP J. Image Video Process., 2013, 2013, (1), p. 38.
-
62)
-
19. Wang, X., Tang, J., Niu, J., et al: ‘Vision-based two-step brake detection method for vehicle collision avoidance’, Neurocomputing, 2016, 173, pp. 450–461.
-
63)
-
6. Wu, N.: ‘High speed CMOS vision chips’. 2011 IEEE 54th Int. Midwest Symp. on Circuits and Systems (MWSCAS), 2011, pp. 1–4.
-
64)
-
30. Hammond, M., Qu, G., Rawashdeh, O.A.: ‘Deploying and scheduling vision based advanced driver assistance systems (ADAS) on heterogeneous multicore embedded platform’. 2015 Ninth Int. Conf. on Frontier of Computer Science and Technology, 2015, pp. 172–177.
-
65)
-
11. Mielke, M., Schafer, A., Bruck, R.: ‘ASIC implementation of a Gaussian pyramid for use in autonomous mobile robotics’. IEEE 54th Int. Midwest Symp. on Circuits and Systems (MWSCAS), 2011, pp. 1–4.
-
66)
-
62. Aeberhard, M., Rauch, S., Bahram, M., et al: ‘Lessons learned from automated driving on Germany's highways’, IEEE Intell. Transp. Syst. Mag., 2015, 7, (1), pp. 42–57.
-
67)
-
71. Saussard, R., Bouzid, B., Vasiliu, M., et al: ‘Towards an automatic prediction of image processing algorithms performances on embedded heterogeneous architectures’. 2015 Int. Conf. on Parallel Processing Workshops (ICPPW), 2015, pp. 27–36.
-
68)
-
51. Butler, R.W., Finelli, G.B.: ‘The infeasibility of quantifying the reliability of life-critical real-time software’, IEEE Trans. Softw. Eng., 1993, 19, (1), pp. 3–12.
-
69)
-
33. Souani, C., Faiedh, H., Besbes, K.: ‘Efficient algorithm for automatic road sign recognition and its hardware implementation’, J. Real-Time Image Process., 2014, 9, (1), pp. 79–93.
-
70)
-
31. Koukoumidis, E., Martonosi, M., Peh, L.: ‘Leveraging smartphone cameras for collaborative road advisories’, IEEE Trans. Mob. Comput., 2012, 11, (5), pp. 707–723.
-
71)
-
40. Chen, D., Lin, Y., Peng, Y.: ‘Nighttime brake-light detection by Nakagami imaging’, IEEE Trans. Intell. Transp. Syst., 2012, 13, (4), pp. 1627–1637.
-
72)
-
32. Giesemann, F., Pay, G., Limmer, M., et al: ‘A comprehensive ASIC/FPGA prototyping environment for exploring embedded processing systems for advanced driver assistance applications’. 2014 Int. Conf. on Embedded Computer Systems: Architectures, Modeling, and Simulation, 2014, pp. 314–321.
-
73)
-
39. Costea, A.D., Vesa, A.V., Nedevschi, S.: ‘Fast pedestrian detection for mobile devices’. 2015 IEEE 18th Int. Conf. on Intelligent Transportation Systems (ITSC), 2015, pp. 2364–2369.
-
74)
-
22. Ozaki, N., Uchiyama, M., Tanabe, Y., et al: ‘Implementation and evaluation of image recognition algorithm for an intelligent vehicle using heterogeneous multi-core SoC’. 2015 20th Asia and South Pacific Design Automation Conf. (ASP-DAC), 2015, pp. 410–415.
-
75)
-
67. Broggi, A., Member, S., Buzzoni, M., et al: ‘Extensive tests of autonomous driving technologies’, IEEE Trans. Intell. Transp. Syst., 2013, 14, (3), pp. 1403–1415.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2016.0026
Related content
content/journals/10.1049/iet-its.2016.0026
pub_keyword,iet_inspecKeyword,pub_concept
6
6