Over the past decades, facial expression recognition (FER) has become an interesting research area and achieved substantial progress in computer vision. FER is to detect human emotional state related to biometric traits. Developing a machine based human FER system is a quite challenging task. Various FER systems are developed by analysing facial muscle motion and skin deformation based algorithms. In conventional FER system, the developed algorithms work on the constrained database. In the unconstrained environment, the efficacy of existing algorithms is limited due to certain issues during image acquisition. This study presents a detailed study on FER techniques, classifiers and datasets used for analysing the efficacy of the recognition techniques. Moreover, this survey will assist researchers in understanding the strategies and innovative methods that address the issues in a real-time application. Finally, the review presents the challenges encountered by FER system along with the future direction.

Currently, the conventional license plate location method fails to detect the license plate under complex road environments such as severe weather conditions and viewpoint changes. Besides, it is difficult for license plate location method based on machine learning to precisely locate the area of license plate. Moreover, license plate location method may incorrectly detect similar objects such as billboards and road signs as license plates. To alleviate these problems, this article proposes a new approach to vehicle license plate location based on new model YOLO-L and plate pre-identification. The new model improves in two aspects to precisely locate the area of license plate. First, it uses k-means++ clustering algorithm to select the best number and size of plate candidate boxes. Second, it modifies the structure and depth of YOLOv2 model. Plate pre-identification algorithm can effectively distinguish license plates from similar objects. The experimental results show that authors’ proposed method not only achieves a precision of 98.86% and a recall of 98.86%, which outperforms the existing methods, but also has high efficiency in real time.

A large-scale multi-projector display system offers high-resolution, high-brightness and immersive visualisation for realistic experience to end users. It has been demonstrated to be effective tackling the conflict between the increasing demands of super-resolution display and the resolution limitation of a single display system. However, there is still no standardisation method for curved-surface projection screen. In this study, we propose a novel approach for calibrating multi-projector display systems, which have curved surfaces. First, based on a detailed analysis on arbitrarily curved surfaces, we present a three-dimensional reconstruction algorithm based on Bezier surface models. Then, for fully utilising the projection area of each projector, we propose a novel curved-surface stitching algorithm to achieve geometry seamlessness of multi-projector display systems. Experimental results show that by constructing local Bessel models for the curved screen, the proposed method performs better than traditional approaches, i.e. the new method achieves geometric calibration with higher accuracy. The proposed method of modelling projection screen and the corresponding automatic geometric correction scheme effectively increase the utilisation ratio of the original projection area of each projector and improve the calibration accuracy of multi-projector system with continuous curved surface.

To address the challenge of repetitive target appearance variation and frequent occlusion, existing visual tracking methods either handle corrupted samples or correct the appearance model. In this study, the authors propose a novel framework that successfully combines these two strategies. In their method, the base tracker is an improved discriminative correlation filter-based tracker, in which an independent classifier is employed to alleviate the problem of corrupted samples; the best model is selected for improvement from a group of models, which they call a ‘model colony’. The model colony is composed of models updated via different processes. The correlation output and the peak-to-sidelobe ratio are used to evaluate each model in the model colony. In addition, they propose a novel criterion called the maximum-to-others ratio for superior model selection. Experiments on 80 challenging sequences show that their tracker outperforms state-of-the-art trackers. In addition, experimental results demonstrate that their formulation significantly improves the performance of their base tracker.

A key challenge in cancer immunotherapy biomarker research is quantification of pattern changes in microscopic whole slide images of tumour biopsies. Drug development requires a correlative analysis of various biomarkers. To enable that, tissue slides are manually annotated by pathologists, which is a tedious and error-prone task. Automation of this annotation process can improve accuracy and consistency while reducing workload and cost. The authors present a deep learning method to automatically segment digitised slide images with multiple stainings into compartments of tumour, healthy tissue, necrosis, and background. The method is based on using a fully convolutional neural network including a colour deconvolution segment learned end-to-end and helping the network to converge faster and deal with the dataset staining variability. They evaluate the performance of the proposed method using the F1 score, which is the harmonic mean between precision and recall. They report a testing F1 score of 0.88, 0.9, 0.8, and 0.99 for tumour, tissue, necrosis, and background, respectively. They address the task in the context of drug development where multiple stains exist and look into solutions for generalisations over these image populations. They also apply visualisation techniques to help understand the network decisions and gain more trust from pathologists.

Recent advances in computation power have allowed computed tomography (CT) to utilise iterative reconstruction (IR) algorithms. The IR technique can handle noisy data and reconstructs optimal CT images from limited projected images. As of cyclic image processing, IR improves the quality of CT images. This approach requires a minimum number of projections to reconstruct an image; however, decreasing the number of projections to 90 can create artefacts and degrade reconstruction quality. To overcome this limitation, the optical flow technique can compute flow vectors between two consecutive projections to generate projected images between frames. Here, optical flow-based frame interpolation combined with the ordered subset-modified iterative technique is proposed to reduce computation time, lower the number of projections, and increase reconstruction quality of CT images. The proposed technique can be used to reconstruct a CT image from 90 projections at 4 degree intervals between projection sequences. This approach produces a much better quality reconstruction compared to that produced by an analytical algorithm, which uses 360 projections. The inclusion of an ordered subset reconstructs CT images quickly by accelerating streaming architecture.

Many histogram equalisations-based methods have been proposed to be applied in contrast enhancement. However, a few methods that can simultaneously create a natural enhancement in images with low, median, and high brightness ranges are suggested. Here, a robust contrast enhancement algorithm, which is called triple clipped dynamic histogram equalisation based on standard deviation (TCDHE-SD), is proposed. In the proposed method, the histogram of the input image is partitioned into three parts with approximately the same number of pixels based on the standard deviation. Then, the process of histogram clipping is performed on each sub-histogram. After that, all histograms will be mapped to a new dynamic range by means of applying some simple calculations, and finally, the sub-histogram equalisation process will be performed independently. This method is proposed to achieve the multiple purposes of the maximum average information content (entropy), controlling the enhancement ratio and preserving reasonable brightness. It also provides a natural enhancement by generating clear images with maximum details. The performance assessment of the proposed method in terms of entropy as well as visual quality based on the mean opinion score (MOS) indicates a significant advantage of it over the previous ones.

This study proposes a new, simple but effective technique to detect and restore colour cast images, named modified grey world method. This method detects colour cast images of outdoor surveillance videos by computing the values in the YUV colour space, which makes it much easier than classic methods. Specific colour cast can be found out by calculating the hue values. Additionally, this method can detect not only simple colour cast images but also multiple colour cast images simultaneously. To detect and restore a colour cast image, the authors first remove all grey pixels and separate it into multiple parts with a maze-solving algorithm. Then, they compute the YUV colour values of each part. If the values are too high or too low, this part of the input image is designated as a colour cast. Finally, they carry out a restoration procedure, in which they calculate weights by matching average colour value with a grey reference value in YUV colour space. This method has been tested in the Safety City surveillance system in Wuhan city, China. The results show that the proposed method leads to better results in detecting and restoring colour cast imaging than classic methods in outdoor surveillance videos.

This paper proposes a novel high-performance multispectral local descriptor that uses orthogonal Walsh codes during the generation of the discriminative feature set. Rotational variance and noise are compelling factors that significantly affect the distinctive performance of a facial identifier. The descriptor proposed in this article handles these challenges favourably sacrificing any distinctive performance. Orthogonal Walsh codes are used during the generation of the local descriptor. A Walsh code is assigned to each neighbour of a reference pixel. Before the assignment of an orthogonal code to each neighbouring pixel, these pixels are sorted in ascending order that consolidates the robustness of the method against rotational variances. Almost all methods proposed so far focus on grayscale images. However, colour bands contain important information about the relationship between pixels. Therefore, authors’ method considers RGB colour bands of pixels to improve distinctive performance. The results of extensive simulations show the remarkable and competitive performance of the proposed method regarding recognition accuracy and robustness against rotational variances as well as noise effects.

Millions of parcels and pieces of luggage are scanned daily for threat detection at border control or high-security buildings. Currently, the process is manually operated by security agents and it is slow and time-consuming. The automation of this process will lift the burden from the security agents and will allow larger volumes of items to be scanned. The authors consider the problem of automatic threat detection, in particular firearms, in X-rays. To achieve this goal, they propose a hybrid algorithm that combines two well-established image segmentation algorithms into a two-step clustering method. The first step is a semi-supervised spectral clustering algorithm at the image level, which classifies whole images into benign or containing a threat. The images classified as threatening from the first step proceed to the second stage, where a variational image segmentation algorithm performs clustering at the pixel level to locate the threat if it exists. The hybrid algorithm is designed to scale-up the processing of hundreds of images, in comparison to the academic literature where only a handful images are used for demonstration. Numerical experiments establish that the combination of two different algorithms produces better results than using individual algorithms.

In the traditional active contour models, global region-based methods fail to segment images with intensity inhomogeneity, and local region-based methods are sensitive to initial contour. In this study, a novel fuzzy energy-based active contour model is proposed to segment medical images, which are always corrupted by intensity inhomogeneity. In order to deal with intensity inhomogeneity, a local energy term is first constructed by substituting a non-local weight for Gaussian kernel widely used in traditional local region-based models. Second, the defined adaptive force can drive the level set function to adaptively increase or decrease according to image intensity information. Therefore, the initial contour can be initialised as a constant function, which eliminates the problem caused by contour initialisation. Moreover, the proposed active contour model is a convex function. Thus, the problem, resulting from optimising a non-convex functional in the traditional active contour models, can be avoided. Experimental results validate the superiorities and effectiveness of the proposed model for image segmentation with comparisons of those yielded by several state-of-the-art techniques.

The conventional masked image restoration algorithms all utilise the correlation between the masked region and its neighbouring pixels, which does not work well for the larger masked image. The latest research utilises Generative Adversarial Networks (GANs) model to generate a better result for the larger masked image but does not work well for the complex masked region. To get a better result for the complex masked region, the authors propose a novel fast GANs model for masked image restoration. The method used in authors’ research is based on GANs model and fast marching method (FMM). The authors trained an FMMGAN model which consists of a neighbouring network, a generator network, a discriminator network, and two parsing networks. A large number of experimental results on two open datasets show that the proposed model performs well for masked image restoration.

In this study, the authors aim to colourise a greyscale image using a fully automated framework which retrieves similar images from a reference database and then transfers the colour from the most similar retrieved images to perform colourisation. Inspired by the recent success of deep learning techniques in extracting semantic information from images, they first use fc7 features from AlexNet to retrieve similar images from the reference database. Top-k retrieved images are considered for colour transfer to the target greyscale image, using various pixel level features. The images which result from the previous step are given a colour enhancement with Reinhard stain normalisation. They follow a pixel-wise colour saturation based averaging technique to impart colour at pixel level. The final image is rectified using joint bilateral filtering. The resulting coloured images have a realistic appearance, similar in quality to the original coloured images. The proposed method outperforms several previous colourisation techniques, yielding superior performance both quantitatively and qualitatively. The method also enhances low-contrast images.

The burrs on high-voltage copper contact leads to point discharge and device damage. Since the high-voltage copper contact has different machining batch and the contour various which the machine tool remove is not economic and robot usually is used to remove the burrs of high-voltage copper contact. The first step for robot deburring is to identify burrs. In order to improve the performance of copper contact burr video denoising, this article presents online burr video denoising sparsifying transforms algorithm, which defined two alternative values between the optimal sparse signal and transform learning dictionary, simultaneously, calculated the mean of peak signal-to-noise ratio, the mean of execution time, the STD (STandard Deviation), and the VAR (VARiance), accordingly presented an burr video denoising algorithm and compared with state-of-the-art video denoising algorithms. The experiment results show that compared with traditional methods, the burr video denoising algorithm has higher denoising precision, faster denoising speed, and stronger high-noise-level processing capacity, and so on. The numeric experiments show that the proposed approach has higher peak signal-to-noise ratio and less computation complexity than the existing video denoising methods.

Small-sized fingerprint sensors, due to the convenience of integration, are widely used in many applications, especially on smart phones. However, the friction ridge information decreases with the reduction of the collected fingerprint area, resulting in degraded recognition performance. Mosaicking fingerprint impressions has been proved to be effective in boosting the recognition accuracy. Nonetheless, the minutiae-based mosaicking methods do not work well when there is no sufficient number of minutiae in the overlapping area while existing minutia-free mosaicking methods are not robust to distortion and result in low mosaicking accuracy. In this study, a novel minutia-free mosaicking algorithm used the coarse-to-fine approach is proposed to obtain a larger fingerprint impression from a couple of small-sized fingerprint impressions. It consists of three stages: an orientation field-based coarse alignment, a ridge matching-based fine alignment, and a nonlinear deformation correction with block-correspondence Thin Plate Spline model. Experimental results on the XDfinger database demonstrate that the proposed method outperforms the other six mosaicking methods in terms of reject-to-fuse rate, registration accuracy, and verification performance. Specifically, in the verification scenario, the equal error rate is reduced from 1.98% of a single impression to 0.41% of two impressions mosaicked by the authors' method.

Industrial baking of sponge cakes requires various quality indicators to be measured during production such as moisture content and sponge hardness. Existing techniques for measuring these properties require randomly selected sponges to be removed from the production line before samples are manually cut out of each sponge in a destructive way for testing. These samples are subsequently processed manually using dedicated analysers to measure moisture and texture properties in a lengthy process, which can take a skilled operator around 20 min to complete per sponge. In this study, the authors present a new, single sensor hyperspectral imaging approach, which has the potential to measure both sponge moisture content and hardness simultaneously. In the last decade, hyperspectral imaging systems have reduced in cost and size and, as a result, they are becoming widely used in a number of industries and research areas. Recently, there has been an increased use of this technology in the food industry and in food science applications and research. The application of this technology in the cake production environment, empowered by sophisticated signal and image processing techniques and prediction algorithms as presented in this study has the potential to provide on-line, real-time, stand-off cake quality monitoring.

Outdoor images taken in foggy weather are not suitable for automation due to low contrast. It is a challenging task to remove fog from images specially when the image contains large sky region. The authors propose dark channel-based single image defogging technique to estimate atmospheric light which represents the amount of luminance in a scene in the absence of fog. This atmospheric light is used to reconstruct fog-free image with a transmission map. Transmission map represents the effect of fog with respect to depth in image. In this study, they propose four transmission maps to reconstruct the images with different colour contrast. Proposed method adaptively selects a transmission map depending upon the fog density to reconstruct image with optimal colour contrast. The transmission map is refined by applying Laplacian filter followed by the guided filter. Previously, dark channel prior based methods were considered to be less effective for images with large sky region, but the proposed method reconstructs better result consistently for such images, independent of the density of the fog. Experimental results show that images reconstructed by proposed method are qualitatively better than the previously proposed methods.

In this study, a reduced-reference image-quality-assessment (IQA) method for screen content images, named as feature-quality-index (FQI) is proposed. The proposed method is based on the fact that the human visual system is more sensitive towards change in features than intensity or structure. Reduced features from the reference and distorted images are first extracted. In order to find the preserved features in the distorted image, a feature matching process with a reduced number of distance calculations is proposed, namely reduced-distance method. To reflect the importance of the matched features and their distance, the inner product between the normalised scale and distance vector is obtained. Extensive comparisons are performed on two available benchmark databases namely SIQAD and QACS, with eight reduced-reference, and nine full-reference state-of-the-art IQA techniques to demonstrate the consistency, accuracy, and robustness of the proposed FQI. The subjective evaluation of mean opinion score shows that FQI outperforms the current state-of-the-art IQA techniques.

Lane departure warning system used in vehicles has recently become very popular and is about to become a vital component in advanced driver assistance systems. The performance of this system is directly related to lane detection accuracy. In this study, a fuzzy inference system-based filter for robust lane detection is proposed. The proposed filter has three input parameters which are as follows: the difference between a pixel and its left and right neighbours at a certain distance along the horizontal direction and standard deviation of the pixels between the left and right neighbours. The parameters of the proposed fuzzy filter are determined in a learning phase by taking challenging scenarios such as varying lighting conditions, shadows, and road cracks. Experimental results reveal that the proposed method outperforms existing lane detection filters when integrated into a lane detection system. Since the proposed approach is computationally lightweight, it is suitable for real-time devices and applications.

In computer vision, the analysis of image contents plays a significant role to perform intelligent tasks such as object recognition and image retrieval. These contents can be low-level visual features or colour information within an image. For content-based image retrieval (CBIR), several methods have been proposed that focus on either low-level visual features extraction or the colour information, and very few works can be seen that retrieve the images by fusing both types of contents. Consequently, this work addresses the problem of combining low-level visual features with colour information that helps to improve the retrieval accuracy of CBIR. The proposed strategy extracts the low-level visual salient features with features from accelerated segment test feature descriptor and quantises the salient keypoints into a feature vector. The colour information of the image is extracted and segmented with non-linear L*a*b* colour space and quantised into a feature vector. The similarity for both the feature vectors including visual and colour features is computed and combined together. The top-rank images are retrieved for the obtained feature vector using the distance metric. The experimental results on two standard benchmark datasets show the improved efficiency and 85% accuracy of the proposed strategy over state-of-the-art methods.

The single image super-resolution (SISR) methods based on the deep convolutional neural network (CNN) have recently achieved significant improvements in accuracy, advancing the state of the art. However, these deeper models are computationally expensive and require a large number of parameters. Accordingly, they demand more memory and are unsuitable for on-chip devices. In this study, a novel SISR method using a deeply recursive CNN with skip connections and a network in network structure is proposed. The deeply recursive CNN with skip connections is adopted for the image feature extraction at both local and global levels. Parallelised 1 × 1 CNNs, usually called a network in network structure, are adopted for image reconstruction. Specifically, recursive learning is utilised to control the number of model parameters needed and residual learning is used to ease the difficulty of training. The proposed method performs favourably against the state-of-the-art methods in terms of computational speed and accuracy. It significantly outperforms the previous methods by a large margin, while demanding far fewer parameters. This model requires less memory and is friendly to on-chip devices.