Visual object tracking is an active topic in the computer vision domain with applications extending over numerous fields. The main sub-tasks required to build an object tracker (e.g. object detection, feature extraction and object tracking) are computationally intensive. Also, the real-time operation of the tracker is indispensable for almost all of its applications. Therefore, complete hardware or hardware/software co-design approaches are pursued for better tracker implementations. This study presents a literature survey of the hardware implementations of object trackers over the last two decades. Although several tracking surveys exist in the literature, a survey addressing the hardware implementations of the different trackers is missing. The authors believe this survey would fill the gap and complete the picture with the existing surveys of how to design an efficient tracker and point out the future directions researchers can follow in this field. They highlight the lack of hardware implementations for state-of-the-art tracking algorithms as well as for enhanced classical algorithms. They also stress the need for measuring the tracking performance of the hardware-based trackers. Additionally, enough details of the hardware-based trackers need to be provided to allow a reasonable comparison between the different implementations.

Lung cancer is one of the leading causes of death in the world. Although early detection of lung tumours (nodules) can remarkably diminish the mortal rate, precise detection of them is not always possible by visual inspection of the computerised tomography images. Since nodules with different sizes have non-uniform shape and brightness, texture attributes and also the gradient of orientation can be good candidate features, which have been used for this purpose. They determined the co-occurrence matrix of the extended local binary pattern (ELBP) along with weighted orientation difference (WOD) for each sub-region of the lung area. Local binary pattern is a texture descriptor that can extract the discriminative features efficiently. The proposed ELBP is rotation invariant and suitable to describe non-uniform patterns. Moreover, WOD as a structural feature uses the magnitude of each edge as the weight of its orientation difference. After constructing the co-occurrence matrix, discriminative features were extracted from this matrix and fed into a support vector machine in order to classify each sub-region as a cancerous (nodule) or normal tissue. The proposed method was compared to some of state-of-the-art nodule detection methods and was assessed over several real datasets in terms of specificity, sensitivity and accuracy.

Display of images on different display devices having varied size and aspect ratio requires one to resize them. Many attempts have been made to perform content-aware image retargeting while generating an image compatible with a target display size. Seam carving is one of the image retargeting operators which alters the size of an image by removing least energy pixels. However, it requires high computational time in order to perform retargeting. In this study, the authors accelerate the naive seam carving process by removal or insertion of multiple pixel wide batch seam in a single iteration rather than a single pixel wide seam. Along with the energy of pixels to be removed, inserted energy after the removal of a batch seam is also minimised in order to prevent the inclusion of false edges. The width of a batch seam is a critical factor which is made adaptive during the retargeting process to preserve the energy of an image. They have shown a significant decrease in computational time with the increase in the width of a batch seam. They have compared the proposed technique with other state-of-the-art image retargeting operators using different quality assessment metrics and visual results.

Histogram equalisation (HE) is a simple and effective image enhancement technique. However, it suffers from excessive brightness change and provides degradation in the visual aspect of the image. To overcome the shortcomings in the HE, a novel histogram framework is proposed in this study. The image histogram is first segmented into two parts using the Otsu's thresholding method. Then, both of the upper and lower histograms are constrained to control the level of enhancement. These constraint parameters are computed through moth-flame optimisation algorithm. After constraining the histograms, mean shift correction is performed to ensure there is a minimum level of mean shifting from input to output image. Traditional HE is then applied with a modified histogram to obtain mapping function for lower and upper grey level individually. This enhanced image provides a balance between the level of enhancement and preservation of the important features of the image for high-level processing. The effectiveness of the proposed method is highlighted with a detailed comparison with other closely related schemes. Through the proposed routine, the enhanced images achieve a good trade-off between features enhancement, low contrast boosting, and brightness preservation in addition to the natural feel of the original image.

Principal component analysis (PCA) is the most widely used unsupervised dimensionality reduction approach. However, most PCA based on the squared reconstruction errors assume that all training samples have been centred, which make them not robust to outliers or noises in the samples and will depress their performance of classification accuracy. On the other hand, when there are various correlations in the training samples, the l ₁-norm regularisation encounters instability problems. To address the above problems, the authors propose a novel L _2,1-norm-based sparse PCA with the trace norm regularised term (abbreviated to OMSPCA-L21-TN) to learn the optimal projection matrix and optimal mean simultaneously, where the objective function in model consists of the L _2,1-norm-based reconstruction error and the trace-norm-based regularised term of the projection vectors involved the sample matrix. Thus, not only can the authors’ method obtain the sparse features and reduce the effect of noise and outliers but also be adaptive to the correlation of the training samples. An effective optimisation solution is also given. The experimental results on some publicly available datasets demonstrate that the proposed approach is feasible and effective.

Most image deblurring methods require large algorithm computational cost because multi-scale blind deconvolution is used for estimating kernel. Furthermore, a moving quick response (QR) code image is regarded as a type of classical blurry image and requires real-time processing in practical applications. Therefore, this study proposes a new framework of motion blurry QR code image restoration in real-time based on the fractional-order deblurring method. The authors perform a trade-off between algorithm computational cost and quality of the deblurring image. First, a black frame is added around the traditional QR code, which is used for locating QR code and reducing the computational cost. Next, a new image deblurring method is proposed using fractional differential order and is used for improving the quality of the deblurring image. Furthermore, an average grey-level method is presented to reconstruct the standard QR code images. Comparisons with the existing algorithms demonstrate that the proposed method can achieve favourable deblurring quality and acceptable computational cost. Finally, their framework is validated in a practical platform of an actual conveyor belt system with a low-cost industrial camera. Experimental results indicate that their framework performs favourably with processing motion blurry QR code images.

Visual saliency is a computational process that seeks to identify the most attention drawing regions from a visual point. In this study, the authors propose a new algorithm to estimate the saliency based on partial difference equations (PDEs) method. A local or non-local graph is first constructed from the geometry of images. Then, the transcription of PDE on graph is done and resolved by using the mean curvature flow that can be used to perform regularisation and the Eikonal equation for segmentation. Finally, an extended region adjacency graph is built, which is extended with a k-nearest neighbour graph, in the mean RGB colour space of each region in order to estimate saliency. The proposed algorithm allows to unify a local or non-local graph processing for saliency computing. Furthermore, it works on discrete data of arbitrary topology. For evaluation, the proposed method is tested on two different datasets and 3D point clouds. Extensive experimental results show the applicability and effectiveness of the proposed algorithm.

This study proposes a new method for simultaneous image segmentation and moderate bias correction. Though many methods are proposed to deal with the image intensity inhomogeneity, some problems still exist and have influenced the segmentation results a lot. In this study, a new model is proposed for image segmentation and correction based on the multiplicative intrinsic component optimization (MICO) model. First, the new model in the level set formulation for gray images has been presented and the split Bregman method for fast minimization has been applied. The proposed model is tested with lots of magnetic resonance images and some medical colour images with promising results. Experimental results show that the proposed model can simultaneously segment images and correct bias field moderately. In the experimental part for gray images, a qualitative comparison between the proposed model and the MICO model in both segmentation and bias-correction results is made. Besides, the proposed model with the Chan-Vese model and the illumination and reflectance estimation model in the experimental part for colour images are compared. Moreover, the proposed model can segment nature colour images successfully. It is clear that the proposed model has a good performance on many characteristics such as accuracy, efficiency, and robustness.

Due to the limitation of existing filters in detection and removal of fixed value impulse noise, the authors propose an iterative grouping median filter (IGMF) according to the characteristics of noise intensity and distribution. It sorts the noise-free pixels in neighbourhood by intensity, divides the sorted pixels into groups depending on the intensity differences of adjacent pixels, and finally takes the median of the maximum group as the intensity of noisy pixel. This noise removal strategy is performed iteratively and takes full advantage of the previous denoising results. Experiments show that IGMF outperforms the existing state-of-the-art filters in terms of visual perception, peak signal to noise ratio and structural similarity index at various noise densities.

Context-based adaptive binary arithmetic coding (CABAC) is a single operation mode for entropy coding in the last video coding standard high-efficiency video coding. For high-resolution applications, the throughput of one bin/cycle is not sufficient and it is a very challenging task to implement pipeline and/or parallel CABAC decoding architecture by simply adding more stages. Indeed, the tight data dependencies make it difficult to parallelise and cause it to be a throughput bottleneck for video decoding. Consequently, in order to improve the CABAC decoder throughput, parallel and pipeline architectures are used in authors’ design. In this work, an algorithm-architecture adequation is proposed to implement a CABAC decoder on a field programmable gate array. Mainly, a new classification of 32 syntax elements is given to speed up the authors’ solution. Furthermore, the context selection and modelling of regular syntax elements are studied, designed and implemented. Finally, a novel technique of memories rearrangement to reduce the critical path delay required to process each binary symbol is proposed. As a result, the implementation can process 2.2 bins/cycle when operated at 123.49 MHz and exhibits an improved high-throughput of 271.678 Mbins/s. The hardware architecture is coded using hardware description language and synthesised using ISE Xilinx tools targeting the Virtex4 platform.

This work proposes an invariant descriptor and a pipeline for the registration of surface range images based on segmentation/reconstruction making use of an edge detection technique combined with a clustering technique using mesh decimation. This novel descriptor is applied to contours and it is invariant to similarity transformations including rotation, translation, uniform scale and it is robust to noise. The proposed feature descriptor makes use of corresponding points extracted from two images and a signature label is assigned specifically to a point considering the geometrical distribution of its neighbourhood, reducing possible areas of overlapping and the ambiguity in the search process. The descriptor was evaluated through a series of tests with various object range images. To validate the candidate transformations, the fitting errors between the two range images are evaluated by the iterative closest point algorithm. This study also presents and discusses results from the application of the developed pipeline in a vision sensor mounted on a robot arm specially built as part of a R&D project to acquire range images by laser scanning over the surface of hydraulic turbine blades. The sensor generates 3D surface models to be registered in the 3D coordinate system of the robot controller.

Here, an extended version of the symmetrical local threshold (SLT) algorithm is introduced for lane feature extraction and used in a novel lane-detection system. The introduced feature map extractor utilises parallel lane border features as well as the dark-light-dark (DLD) pattern of the lane marking used in SLT. Hence, compared to the SLT, the true positive to positive rate of the calculated feature maps is increased from 69% to 86% on the ROMA dataset. In addition, the proposed algorithm supplies orientation information for the estimated feature points, which can be useful for many optimisation algorithms. Consequently, based on the estimated lane feature orientations, a global lane orientation is calculated and used for both enhancing the feature map and estimating a one-dimensional (1D) lateral offset likelihood function. Then, the estimated 1D functions are filtered temporally and up to two linear lane candidates are detected. For increased flexibility, robust fitting is applied to the feature points in the region of interest (ROI). Finally, based on the detection of the previous frame, a mask is created and applied to the next frame. When tested on 2301 road images, mean error in lateral offset is calculated as 4.1 pixel on the IPM images.

Human action recognition (HAR) is a very challenging task because of intra-class variations and complex backgrounds. Here, a motion history image (MHI)-based interest point refinement is proposed to remove the noisy interest points. Histogram of oriented gradient (HOG) and histogram of optical flow (HOF) techniques are extended from spatial to spatio-temporal domain to preserve the temporal information. These local features are used to build the trees for the random forest technique. During tree building, a semi-supervised learning is proposed for better splitting of data points at each node. For recognition of an action, mutual information is estimated for all the extracted interest points to each of the trained class by passing them through the random forest. The proposed method is evaluated on KTH, Weizmann, and UCF Sports standard datasets. The experimental results indicate that the proposed technique provides better performance compared to earlier reported techniques.

Mobile terminal gesture recognition is an extreme challenge, not only because its limited computing resource make it complicated to identify feature points but also the complex background can easily affect the recognition result. This study proposes a gesture recognition method based on improved features from accelerated segment test (FAST) corner detection. First, in order to eliminate the effects of complex background and light, the intersection of the two frame images is obtained through background subtraction and the multi-colour space to realise the detection of the hand. Second, in order to improve the performance of the algorithm, an improved FAST corner detection method combined with the back propagation neural network (BPNN) is proposed in accordance with the characteristics of fingertips. Subsequently, the feature points are screened by method of non-maximum suppression. Finally, gesture recognition is realised by matching feature points. Experimental results illustrate that this method has strong anti-interference ability in complex background, and it is good at performance.

Ultrasound (US) image segmentation plays a very important role in diagnostic imaging. In order to extract special tissues from images, this study proposes a new method for segmentation of US images. The proposed method uses multilevel threshold segmentation which is based on Otsu and differential search algorithm. Testing in simulation US images shows that the proposed algorithm has a better result of segmentation than the three existing methods, including region growing, the active contour model and k-means technique. The proposed method gets the highest F _m values and the smallest area errors in experiments. Vivo US images are also tested by the proposed method and it achieves a good segmentation result.

Owing to their large field of view, a catadioptric imaging system is widely used in several research areas, including robot navigation, surveillance, and three-dimensional (3D) reconstruction. The modelling and calibration of these systems allow the accurate measurement of objects. This paper presents a calibration method for a panoramic 3D reconstruction system based on pattern projection composed of two modules. The first module, which contains a charged coupled device camera and a parabolic mirror allows the acquisition of catadioptric images. The second module is a catadioptric projection system made of a light projector and a parabolic mirror that generates a projected pattern over the target for reconstruction purposes. Both the modules are calibrated independently. The method achieves a reprojection error of 0.6 pixels in the camera module and 1.2 pixels in the catadioptric projection system. To evaluate the performance of the proposed method, the results show 3D reconstructions of multiple objects with tolerances around 1.3 mm.

Semantic segmentation of remote sensing ship targets is one of the most challenging works in image processing, especially for small and multi-scale ship target detection. To solve these problems, an efficient method based on convolutional neural networks (CNN) to detect ship targets is proposed. This method introduces the attention model to the network to enhance the characteristics of small targets and combines atrous convolution with traditional CNN to increase the receptive field. To preserve the information lost by pooling, the proposed method uses the passthrough layer method to retain more features and concatenate the high- and low-resolution features. To verify the effectiveness of the method proposed in this study, the performance was evaluated by using precision, recall, F1-Score, mean intersection-over-union (IoU), and pixel accuracy measurements. These performances are all higher than the traditional semantic segmentation network SegNet. Mean IoU increases to 0.783 and pixel accuracy increases to 0.935. This method can conclusively identify ship targets in remote sensing images and has a certain reference value for remote sensing target detection.

Wireless capsule endoscopy (WCE) has revolutionised the diagnosis and treatment of gastrointestinal tract, especially the small intestine which is unreachable by traditional endoscopies. The drawback of the WCE is that it produces a large number of images to be inspected by the clinicians. Hence, the design of a computer-aided diagnosis (CAD) system will have a great potential to help reduce the diagnosis time and improve the detection accuracy. To address this problem, the authors propose a CAD system for automatic detection of ulcer in WCE images. Firstly, they enhance the input images to be better exploited in the main steps of the proposed method. Afterward, segmentation using saliency map-based texture and colour is applied to the WCE images in order to highlight ulcerous regions. Then, inspired by the existing feature extraction approaches, a new one has been proposed for the recognition of the segmented regions. Finally, a new recognition scheme is proposed based on hidden Markov model using the classification scores of the conventional methods (support vector machine, multilayer perceptron and random forest) as observations. Experimental results with two different datasets show that the proposed method gives promising results.