Imaging and Sensing for Unmanned Aircraft Systems: Volume 2: Deployment and Applications
2: ECE Department, Karunya University, Coimbatore, India
3: Instituto Tecnológico de Aeronáutica, Sao Jose dos Camp, Brazil
4: Robotics and Automation Department of Computer Science Electrical and Space Engineering, Lulea University of Technology, Lulea, Sweden
5: Aerospace Engineering and Aviation, RMIT University, Melbourne, VIC, Australia
This two volume book set explores how sensors and computer vision technologies are used for the navigation, control, stability, reliability, guidance, fault detection, self-maintenance, strategic re-planning and reconfiguration of unmanned aircraft systems (UAS). Volume 1 concentrates on UAS control and performance methodologies including Computer Vision and Data Storage, Integrated Optical Flow for Detection and Avoidance Systems, Navigation and Intelligence, Modeling and Simulation, Multisensor Data Fusion, Vision in Micro-Aerial Vehicles (MAVs), Computer Vision in UAV using ROS, Security Aspects of UAV and Robot Operating System, Vision in Indoor and Outdoor Drones, Sensors and Computer Vision, and Small UAVP for Persistent Surveillance. Volume 2 focuses on UAS deployment and applications including UAV-CPSs as a Testbed for New Technologies and a Primer to Industry 5.0, Human-Machine Interface Design, Open Source Software (OSS) and Hardware (OSH), Image Transmission in MIMO-OSTBC System, Image Database, Communications Requirements, Video Streaming, and Communications Links, Multispectral vs Hyperspectral Imaging, Aerial Imaging and Reconstruction of Infrastructures, Deep Learning as an Alternative to Super Resolution Imaging, and Quality of Experience (QoE) and Quality of Service (QoS).
Inspec keywords: image processing; multimedia systems; cyber-physical systems; wireless sensor networks; autonomous aerial vehicles; quality of service
Other keywords: UAV cyber-physical system; mechatronic framework; communication protocol layers; drone; high-dimensional sensor data; unmanned aerial vehicle; wireless sensor networks; quality of service; wireless multimedia sensor network; WSNs; ground control station; multi-dimensional sensors; wirelessly connected sensor nodes; visual sensor and actuator networks; remotely piloted aircraft system; WMSN; aeronautics; unmanned aircraft system
Subjects: Mobile robots; Optical, image and video signal processing; Wireless sensor networks; Multimedia communications; General and management topics; Aerospace control; Multimedia; General electrical engineering topics; Computer vision and image processing techniques
- Book DOI: 10.1049/PBCE120G
- Chapter DOI: 10.1049/PBCE120G
- ISBN: 9781785616440
- e-ISBN: 9781785616457
- Page count: 277
- Format: PDF
-
Front Matter
- + Show details - Hide details
-
p.
(1)
-
1 UAV-CPSs as a test bed for new technologies and a primer to Industry 5.0
- + Show details - Hide details
-
p.
1
–22
(22)
The extensive Cloud pool of resources and infrastructure can deliver significant improvements to Unmanned Aerial Vehicle (UAV) Cyber-Physical Systems (UAV-CPSs) relying on data or code from a network to operate, but not all sensors, actuators, computation modules and memory depots from a single fixed structure. This chapter is organised around the potential benefits of the Cloud: (1) Big Data (BD) access to visual libraries containing representations/descriptive data, images, video, maps and flight paths, (2) Cloud Computing (CC) functionalities for Grid Computing (GC) on demand for statistical analysis, Machine Learning (ML) algorithms, Computational Intelligence (CI) applications and flight planning, (3) Collective UAV Learning (CUL) where UAVs share their trajectories, control guidelines and mission outcomes and (4) human-machine collaboration through crowdsourcing for analysing high-dimensional high-resolution (HDHR) images/ video, classification of scenes/objects/entities, learning and error correction/ concealment. The Cloud can also expand UAV-CPSs by offering (a) data sets, models, all sorts of literature, HDHR benchmarks and software/hardware simulators, (b) open competitions for UAV-CPS designs with Open Source Hardware (OSH) and (c) Open-Source Software (OSS). This chapter talks about some open challenges and new trends in UAV-CPSs.
-
2 UAS human factors and human–machine interface design
- + Show details - Hide details
-
p.
23
–48
(26)
The human-machine interface (HMI) is a crucial yet often overlooked aspect in the design of Unmanned Aircraft Systems (UASs). A properly designed HMI enhances situational awareness and reduces the workload of the ground pilot, thereby contributing to improving the overall mission performance. Typically, a Human Factors Engineering (HFE) program provides a methodological process to support good design. The program comprises three iterative stages: requirements analysis and capture, design and evaluation. A number of approaches can be adopted in the HFE program but given the wide range of applications and missions that are being undertaken by different types of UAS, it is advantageous to adopt a functional approach towards HMI design, where the HMI is designed around specific functions to be performed by either the human user or the system. The typical UAS functions include mission planning, sensor operation, data analysis and sense-and-avoid (SAA), and can also extend to multi-platform coordination and collaborative decision-making. The human factors considerations and the associated HMI elements supporting these functionalities are discussed.
-
3 Open-source software (OSS) and hardware (OSH) in UAVs
- + Show details - Hide details
-
p.
49
–66
(18)
The popularity of the Open Source Tool (OST) has expanded significantly. This is the case for Unmanned Aerial Vehicles (UAVs) based on open-source hardware (OSH) as well. Open-source software (OSS) and OSH can be applied in a wide range of applications and can improve several technologies. The chapter begins with an introduction to OSS depicting its rationale, description of fundamental differences between OSS and proprietary software (PS), what benefits OSSs provide to overall UAV community, the motives leading people to pick up an OSS instead of a PS, which helps the academic and research community. This chapter also covers some OSSs used within the UAV community to support all aspects of UAV technology and the Remote Sensing (RS) and photogrammetry data post-processing chain. It is possible to build fully autonomous and operational UAV based only on OSH and OSS. The chapter describes the state of the art for OSS widely used in UAV technology, the software used in all aspects of UAV technology such as ARDUPILOT-based Autopilot firmware, MISSION PLANNER-based ground station, OPENTX transmitter software, MINIM On-Screen Data (OSD) software, Open Drone Map photogrammetry data processing suite, Web drone data-processing suite WebODM. This chapter describes several concepts and characteristics of open software/hardware, built-in functions, and particular features as well as platform requirements. A typical UAV photogrammetry workflow for drone construction with flight planning/execution and OSS data processing is provided.
-
4 Image transmission in UAV MIMO UWB-OSTBC system over Rayleigh channel using multiple description coding (MDC)
- + Show details - Hide details
-
p.
67
–90
(24)
Orthogonal Space-Time Block Codes (OSTBC) and multiple-input-multiple-output (MIMO) communication system are new techniques with high performance that have many applications in wireless telecommunications. This chapter presents an image transfer technique for the unmanned aerial vehicle (UAV) in a UWB system using a hybrid structure of the MIMO-OSTBC wireless environment in multiple description coding (MDC) deals. MDC technique for image transmission is a new approach in which there is no record of it so far. This ensures that in the packet loss scenario due to channel errors, images with acceptable quality with no need for retransmission can be reconstructed. The proposed system is implemented using a different number of transmitter and receiver antennas UAV. Assuming a Rayleigh model for the communication channels, the MDC image transmission performance is compared with single description coding (SDC). Experimental results confirm that the proposed hybrid method has better performance than the SDC.
-
5 Image database of low-altitude UAV flights with flight condition-logged for photogrammetry, remote sensing, and computer vision
- + Show details - Hide details
-
p.
91
–111
(21)
The growth in the number of aerial images available is stimulating research and development of computational tools capable of extracting information from these image databases. However, developing a new computer vision (CV) software is complicated because many factors influence the extraction of information from aerial images, such as lighting, flight altitude, and optical sensors. The CV has been incorporated in most modern machines such as autonomous vehicles and industrial robots. The aim is to produce a high-quality image database of low-altitude Unmanned Aerial Vehicle (UAV) flights with flight condition-logged for photogrammetry, remote sensing, and CV. This work resulted in a collection of aerial images in the visible and thermal spectrum, and this set of images was captured in different schedules of the day, altitudes of flight, times of the year. The cameras are synchronised with the UAVs autopilot, and they were spatially and spectrally characterised in the laboratory. This research makes available low altitude aerial images of a region in Brazil to all community, with the precise flight and capture information, as well as additional features such as ground truth and georeferenced mosaic. Examples of the use of the database are shown for mosaic generation and development of CV algorithms for autonomous navigation of UAVs [1,2]. Furthermore, this database will serve as a benchmark for the development of the CV algorithms suited for autonomous navigation by images.
-
6 Communications requirements, video streaming, communications links and networked UAVs
- + Show details - Hide details
-
p.
113
–132
(20)
Unmanned Aerial Vehicles (UAVs) within Cyber-Physical Systems (CPSs) depend on Flying Ad -Hoc Networks (FANETs) as well as on Computer Vision (CV). The Flying Nodes (FNs) play a paramount role in UAV CPSs because relying on imagery poses austere and diverse restrictions in UAV communications. Nowadays, UAV technology is switching from a single UAV to coordinated UAVs swarms that undertake advanced objectives. This scenario calls for innovative networking paradigms to handle two or more FNs that exchange data (i) straightly (without intermediates within their communication range) or (ii) indirectly via relay nodes like UAVs. Designing a UAVs' ad -hoc network is intricate because FANET's prerequisites differ from Mobile Ad -hoc Networks (MANETs) as well as Vehicular Ad -hoc Networks (VANETs). FANETs have particular specificities about FN mobility, FN connectivity, data routing, cloud interaction, Quality of Service (QoS), type of application, and Quality of Experience (QoE), among other issues. This chapter goes through the UAVs' challenges when functioning as ad -hoc nodes and expounds UAVs' network models. It also typifies FANETs' emergent prospects, impact, and how they fit in multimedia applications.
-
7 Multispectral vs hyperspectral imaging for unmanned aerial vehicles: current and prospective state of affairs
- + Show details - Hide details
-
p.
133
–155
(23)
Over the past few decades of imaging, these sensing instruments are now more advanced with multiple missions such as surveillance, monitoring, tracking and destruction of spatial objects. Nowadays, unmanned aerial vehicles (UAVs) are much prevalent, which could acquire a comprehensive view and could perform actions even to the lowest target levels at the ground. The UAV can be developed with minimal cost than other remote mission. Hence, it is much cost-effective. This chapter aims at detailing the critical aspects of two different variants of remotesensing (RS) technologies in UAVs: (a) multispectral imaging (MSI) and (b) hyperspectral imaging, which accounts for the spatial and spectral signatures of the observed underlying natural phenomena.
-
8 Aerial imaging and reconstruction of infrastructures by UAVs
- + Show details - Hide details
-
p.
157
–176
(20)
This chapter presents a compilation of experimental field trials aiming vision-based reconstruction of large-scale infrastructures using micro aerial vehicles (MAVs). The main focus of this study is on the sensor selection, the data-set generation and on the computer vision algorithms for generating three-dimensional (3D) models. In general, MAVs are distinguished for their ability to fly at various speeds, to stabilise their position and to perform manoeuvres close to large-scale infrastructures. The aforementioned merits constitute aerial robots a highly paced evolving robotic platform for infrastructure inspection and maintenance tasks. Different MAV solutions with task-oriented sensory modalities can be developed to address unique tasks, such as 3D modelling of infrastructures. In this chapter, aerial agents navigate around/ along different environments, while collecting visual data for post-processing using structure from motion (SfM) and multi-view stereo (MVS) techniques to generate 3D models [1,2]. The proposed framework has been successfully experimentally demonstrated in real indoor, outdoor and subterranean environments.
-
9 Deep learning as an alternative to super-resolution imaging in UAV systems
- + Show details - Hide details
-
p.
177
–213
(37)
This chapter proposes a framework to super-resolve the low-resolution (LR) images captured using the unmanned aerial vehicle. The framework used a convolution neural network to super-resolve the LR image. This framework also removes the haze present in the LR image. The proposed system is evaluated using peak signal to noise ratio, structural similarity (SSIM) and visual information fidelity (VIFP) in the pixel domain. The experimental results demonstrate the advantage of the proposed method when compared to other state-of-the-art algorithms based on qualitative and quantitative analysis. Future trends in super-resolution (SR) unmanned aerial vehicle (UAV) imaging are discussed at the end of this chapter, followed by the concluding section.
-
10 Quality of experience (QoE) and quality of service (QoS) in UAV systems
- + Show details - Hide details
-
p.
215
–245
(31)
This chapter studies both the quality of service (QoS) and quality of experience (QoE) of unmanned aerial vehicle cyber-physical systems (UAV-CPSs). These parameters help to gather data about the connectivity options in networks containing flying nodes (FNs), ground stations (GSs) and other associated devices. They also support to solve complications related to the number of choices of subnetworks complicates designs, capacity, spectrum efficiency, network coverage and reliability among other issues from a flying ad hoc network (FANET) especially when there is streaming. QoS and QoE permit the discovery of the best conceivable network configurations, and costs for user applications autonomously. Existing lines of attack are listed by function. Restrictions and strong points are emphasised to arrange for initial investigations as well as further studies in this area. If the UAV-CPS network has low QoS, then real-time data will not be accurate and subject to data losses causing inaccuracies. Information loss or delay (due to packet loss, rearrangement and delay) may lessen the satisfaction level of the UAV-CPS user (operator). Network QoS degradation affects the real-time video monitoring and must always be taken into account with on-board needs. This effect leads to data loss and image quality reduction, thus decreasing the QoE.
-
11 Conclusions
- + Show details - Hide details
-
p.
247
–248
(2)
The current awareness in unmanned aerial vehicles (UAVs) has prompted not only military applications but also civilian uses. Aerial vehicles' requirements aspire to guarantee a higher level of safety comparable to see-and-avoid conditions for piloted aeroplanes. The process of probing obstacles in the path of a vehicle and determining whether they pose a threat, alongside measures to avoid these issues, is known as see and avoid or sense and avoid. Other types of decision-making tasks can be accomplished using computer vision and sensor integration since they have a great potential to improve the performance of the UAVs. Macroscopically, UAVs are cyber-physical systems (CPSs) that can benefit from all types of sensing frameworks, despite severe design constraints, such as precision, reliable communication, distributed processing capabilities and data management. This book is paying attention to several issues that are still under discussions in the field of UAV-CPSs. Thus, several trends and needs are discussed to foster criticism from the readers and to provide further food for thought.
-
Back Matter
- + Show details - Hide details
-
p.
(1)