Underwater images are susceptible to various distortions compared to images taken on land, due to the nature of the water environment. These images often suffer from diffraction, polarisation, absorption, scattering, colour loss and attenuation of light. Each part of the ocean will have its own sources of distortions, due to flickers caused by direct sunlight, marine snow, the fluorescence of biological objects, the presence of macroscopical organisms, loss of stability in divers, loss of light, artificial lighting and floating dust particles present in the water. There are numerous techniques and algorithms that may be used to restore these underwater images. This study reviews different algorithms and methods, developed in the past two decades, to give clearer ideas on the techniques present in the image restoration process, specifically for underwater images.

The authors propose here to overcome lacks of robustness against noise and adaptability to image features for which classical morphological operators suffer from. For doing this, they propose to deal with partial differential equations (PDEs) for generalised Cauchy problems, and they show that the proposed PDEs are equivalent to impose both robustness and adaptability to structuring functions of the corresponding sup-inf operators. This allows them to introduce spatially adaptability in levellings, and it turns out that the proposed approach constitutes a PDE formulation and a generalisation of a larger class of levellings, the so-called extended levellings, for which one of them are characterised by quasi-flat zones. They show the efficiency of the proposed approach on synthetic, grey, and colour images with different types of noises.

In the design of three-dimensional image processing systems, stereoscopic image quality assessment (SIQA) plays an indispensable role as a performance evaluator and a supervisor, yet the study upon which remains immature due to the complexity of the human visual system (HVS). In this study, a novel SIQA method is proposed by extracting quality-aware image features according to the properties of the hierarchical structure in the HVS. Especially, the interests of the primary and secondary visual cortex are taken into consideration, so that the image quality representation is constructed in a way both accurate and efficient. Moreover, influences caused by binocular effects including binocular rivalry and binocular visual discomfort are accounted to further improve the performance of the proposed method. The superiority of the proposed method is validated through experiments on public databases in comparison to state-of-the-art works in terms of accuracy and robustness.

Salient object detection (SOD) has been attracting a lot of interest, and recently many computational models have been developed. Here, the authors formulate a SOD model, in which saliency map is computed as a combination of the colour, its distribution-based saliency and orientation saliency. Similar to traditional SODs, the proposed method is based on super-pixel segmentation and super-pixel utilises both colour and its distribution-based saliency to generate a coarse saliency map. However, distinct from traditional SODs, authors further use orientation contrast to optimise the coarse saliency map to obtain an improved saliency map. Authors’ contributions are twofold. First, the authors combine colour uniqueness and its distribution with local orientation information (LOI) used in Itti's model to effectively improve profiles of salient regions. Second, a reciprocal function is defined to substitute the Gabor function used in LOI, and the authors have proved that the substitution could detect relatively homogeneous and uniform regions at the boundary of salient object, whereas it is what the traditional models lack. Authors’ approach significantly outperforms state-of-the-art methods on four benchmark datasets, while the authors demonstrate that the proposed method runs as fast as most existing algorithms.

Band selection of hyperspectral images is an optimal feature selection method, which aims at reducing the computational burden associated with processing the whole data. The significant and informative bands identified by the band selection process lead to efficient, compact representation of the image data and produce a satisfactory performance in the succeeding applications viz. classification, unmixing, target detection and so on. In this study, the authors present an unsupervised manifold clustering approach for band selection, which accounts for different types of scenarios. Unlike other band selection approaches, the authors’ proposed manifold clustering framework identifies the informative bands by utilising the interrelation between the bands and accounts for the multi-manifold structure prevalent in some real images. The proposed band selection framework identifies the optimal number of clusters by cluster validity index, clusters the bands by manifold clustering and select representative bands from each cluster according to graph weight. Their proposed manifold clustering approach is a generic clustering approach, which produces a satisfactory result even when the data contains non-linearity. The information theoretic performance measures, classification and unmixing performance on real image experiments demonstrate the proficiency of their proposed band selection algorithm.

Many pieces of research have been conducted on image-restoration techniques to recover high-quality images from their low-quality versions, but they usually aim to handle a single degraded factor. However, captured images usually suffer from various degradation factors, such as low resolution and compression distortion, in the procedures of image acquisition, compression, and transmission simultaneously. Ignoring the correlation of different degraded factors may result in the limited efficiency of the existing image-restoration methods for captured images. A joint deep-network-based image-restoration algorithm is proposed to establish a restoration framework for image deblocking and super-resolution. The proposed convolutional neural network is made up of two stages. A deblocking network is constructed with two cascade deblocking subnets first, then, super-resolution is performed by a very deep network with skipping links. Cascading these two stages forms a novel deep network. An end-to-end training scheme is developed, which makes the two stages be trained jointly so as to achieve better performance. Intensive evaluations have been conducted to measure the performance of the authors’ method both in general images and face images. Experimental results on several datasets demonstrate that the proposed method outperforms other state-of-the-art methods, in terms of both subjective and objective performances.

Discriminative correlation filter-based tracking algorithms have recently shown impressive performance on benchmark data sets. However, visual tracking is still a challenging task in the case of partial occlusions, irregular deformations and so on. In this study, the authors intend to solve these issues by introducing the adaptive collaborative model into part-based tracking. First, instead of a simple linear superposition, the collaborative strategy they proposed combines the template model and colour-based model adaptively and relies on the strengths of both to promote the accuracy. Second, we utilise the voting strategy to figure out the final object position from reliable parts, and the motion information is used in evaluation for reliable parts to enable the tracker to be robust in various situations. Third, the authors utilise a discriminative multi-scale estimate method to solve the problem of scale variations. Finally, they introduce a dimensionality reduction method to limit the computational complexity of the tracker. Abundant experiments demonstrate that the tracker performs superiorly against several advanced algorithms on both the Online Tracking Benchmark (OTB) 2013 and OTB2015 data sets while maintaining the high frame rates.

In this study, an image enhancement algorithm based on the modified histogram clipping scheme using a difference of histogram bins (MCDHB) has been proposed. The core idea of the proposed method is to ascertain the difference between the number of pixels’ in histogram bins of an input image and that of the traditional histogram equalised (HE) image. The calculated difference of each bin is partitioned into different blocks based on range criteria. The proposed algorithm can be attested as a global HE approach and mainly focuses on maintaining peaks in the histogram. The proposed MCDHB framework provides a good trade-off among contrast enhancement, shape of histogram, detailed information, and natural colour. Furthermore, the MCDHB framework is also incorporated with gamma correction for further improvement. The subjective and objective assessment confirms that both the proposed techniques can efficiently enhance the images, in a better way than those produced by classical techniques.

Hyperspectral imaging has great achievements in agriculture, astronomy, surveillance, and so on. However, the inherent low spatial resolution of hyperspectral imaging, unfortunately, limits its more widespread applications. Recently, hyperspectral image (HSI) super resolution addresses this problem by fusing a low spatial resolution HSI (LR-HSI) with a high spatial resolution multispectral image (HR-MSI), but most of these methods did not consider real-time restoration of high spatial resolution HSI. In this study, the authors propose a fast HSI super-resolution method which fills this blank. Specifically, they model the hyperspectral super resolution as a linear regression problem according to the fact that the imaging process is a linear transform and the inverse of this transform can be approximately estimated, as the spectra of a typical scene lie in a very low-dimensional space. To further exploit the low-dimensional nature of the spectra, they divide the HR-MSI and LR-HSI into several patches and learn the inverse transform patch-by-patch. Experiments on several public datasets show that their method approximates state-of-the-art methods in accuracy, but is several orders of magnitude faster than all of them. Furthermore, they provide an efficient C language implementation of their methods, which can meet the real-time request.

Medical images usually are of low contrast in nature and have poor visual perception capability. Image enhancement techniques can improve significant features such as edge, texture and contrast which are helpful for further processing. This study discusses an image fusion-based enhancement scheme suitable for enhancing liver and lesions from abdominal radiology images. Apart from other fusion techniques, feature-based fusion is employed. The pixel-wise features selected are intensity values, gradient magnitude and local homogeneity. These pixel-wise features are clustered and classified using fuzzy C means (FCMs) and support vector machine (SVM), respectively. FCM clusters pixel-wise features into foreground and background, edge and non-edge as well as homogeneous and non-homogeneous regions. These two classes are applied for training and testing the SVM. The classifier output is transformed into images and the pixel-wise features of these images are fused to form a new image. Another important aspect of this scheme is the fusion of pixel-wise features in three dimensions to form a new image. The resulting image is an RGB image having better visual perception capacity having both enhancement in edge and texture. Pixel level multi-dimensional fusion is capable of enhancing the maximum relevant information.

Aiming at the problem that optical coherence tomography (OCT) images with low contrast and layer structure blur are difficult to be automatically layered, a new OCT detection method based on complex shearlet transform is proposed. The method utilises nearly optimal sparse approximation singular curves of multi-scale shearlet transform, and the contrast invariance of the phase congruence method. Compared with the Canny edge detector and wavelet methods, the complex shearlet-based method achieved the highest Pratt figure of merit (PFOM) value. The PFOM value of a step type edge is 0.92, and that of a pulse type edge is 0.98. Three types of OCT images were tested, including normal retinal macula area, dry age-related macular degeneration, and Stargardt disease. The experimental results show that the complex shearlet-based method can detect more layered structures of OCT images, especially the boundary between the ganglion cell layer and the inner plexiform layer that is difficult to detect, and it can detect various types of OCT images. The complex shearlet-based transform method provides an effective and general way to measure retinal OCT images.

In this article, a region-division-based joint sparse representation classification (RDJSRC) method is proposed to solve the heterogeneous region problem in the joint sparse representation classification (JSRC) method used in hyperspectral image (HSI) classification. The RDJSRC method incorporates regional information, obtained by the hidden Markov random field (HMRF), into the JSRC to reduce the interference of heterogeneous pixels in the neighbourhood of the test pixel and finally improve the classification performance. The framework of this method is as follows. The first several principal components (PCs) are initially selected to be the new HSI by transforming the original HSI with the PC analysis algorithm. Then, the regional information containing the spatial structure of the HSI is obtained by applying the HMRF algorithm to the first PC. Through incorporating this regional information into the JSRC procedure, the initial label of the test pixel can be jointly determined by the new HSI pixels within the homogeneity in the search window. Ultimately, the final label of the test pixel is determined by a voting strategy based on multiple classification results. Compared with several classification methods, experimental results, indicate that this method achieves improvement from 2 to 3% in HSI classification.

Advances in signal processing in the encrypted domain and cloud computing have given rise to privacy-preserving technologies. In recent years, reversible data hiding in encrypted images (RDH-EI) has received attention from the research community because additional data can be embedded into an encrypted image without accessing its original content, and the encrypted image can be losslessly recovered after extracting the embedded data. Although the recent development of RDH-EI compatible with homomorphic public key cryptosystems has intensified research interest, most of the existing mature RDH schemes cannot be transplanted to the encrypted domain due to the limitations of the underlying cryptosystems. In this paper, prediction error expansion based RDH-ED using probabilistic and homomorphic properties of the Paillier cryptosystem is presented. This work implements non-integer mean value computation in the encrypted domain without any interactive protocol between the content owner and the cloud server. This work presents mathematical detail of pixel prediction (mean), prediction error, error expansion and data embedding in the encrypted domain and data extraction and content recovery in the plain domain. Experimental results from standard test images reveal that the proposed scheme outperforms other state-of-the-art encrypted domain schemes.

The level set method based on bias correction can segment images with gentle intensity inhomogeneity effectively. However, most level set methods fail to segment severe inhomogeneous images due to the use of fixed scale clustering criterion. To deal with this problem, an adaptive multilayer level set method is proposed to segment images with severe intensity inhomogeneity. First, an improved global adaptive scale operator and a local adaptive scale operator are designed to adaptively adjust the scale of clustering kernel function according to the degree of intensity inhomogeneity. Then, an adaptive multilayer level set structure is constructed with the two designed scale operators. The number of layers and the scale of each layer in the multilayer structure are adaptively determined based on the degree of intensity inhomogeneity, which not only provides appropriate candidate scales in each pixel but also allows the model to detect global contrast information. With the dual minimisation method, image segmentation and bias correction can be achieved simultaneously. In addition, a hybrid bias field initialisation procedure is proposed to enhance the robustness of the proposed method. Experimental results demonstrate the effectiveness and robustness of the proposed method in segmenting images with intensity inhomogeneity.

In cardiac imaging, the boundary of the left ventricle (LV) could be used to measure the dyssynchrony of the heart. Hence, automatic and optimal segmentation of the LV is important. Although deep learning-based methods have achieved significant break-throughs in the accuracy of segmenting LV, it relies on a great number of training sets and the reproduction quality of the tested cases. Due to the variety of patients, it is difficult or impossible to collect the complete training sets that cover all patients with different genders, races, and ages. Therefore, methods independent of the training sets are more reliable and efficient for clinical applications. In this study, the authors propose a training sets-independent method to segment LV optimally and it outperforms all available state-of-the-art training-sets-independent image segmentation methods. In addition, they propose a framework to identify the boundary of the LV automatically. They tested these segmentation methods with both good quality and poor quality images in the proposed framework and verified that the proposed segmentation method yields the optimal solution compared to other state-of-the-art training-sets-independent segmentation methods. Based on their previous research work, the identified boundaries by the proposed approach are accurate enough for calculating the dyssynchrony of the LV.

Medical image quality requirements have been increasingly stringent with the recent developments of medical technology. To meet clinical diagnosis needs, an effective medical image enhancement method based on convolutional neural networks (CNNs) and frequency band broadening (FBB) is proposed. Curvelet transform is used to deal with medical data by obtaining the curvelet coefficient in each scale and direction, and the generalised cross-validation is implemented to select the optimal threshold for performing denoising processing. Meanwhile, the cycle spinning scheme is used to wipe off the visible ringing effects along the edges of medical images. Then, FBB and a new CNN model based on the retinex model are used to improve the processed image resolution. Eventually, pixel-level fusion is made between two enhanced medical images from CNN and FBB. In the authors’ study, 50 groups of medical magnetic resonance imaging, X-ray, and computed tomography images in total have been studied. The experimental results indicate that the final enhanced image using the proposed method outperforms other methods. The resolution and the edge details of the processed image are significantly enhanced, providing a more effective and accurate basis for medical workers to diagnose diseases.

To achieve an automatic and accurate segmentation of lungs and improve the clinical efficiency of computer-aided diagnosis, the authors present a lung segmentation algorithm based on the random forest method and a multi-scale edge detection technique. The algorithm carries a first step of lung region extraction and a second step of lung nodule segmentation. By combining texture information, the improved superpixel generation method can better deal with initial segmentation on lung computed tomography images with inhomogeneous intensity. Then, the lung region is further extracted by using the random forest classifier on the superpixel features, and the lung contours are corrected with a proposed circle tracing technique. Finally, the segmentation is further refined by employing a multi-scale edge detection technique, which enables their method to detect suspicious nodules with various intensities and sizes adaptively. The effectiveness of the proposed approach is demonstrated on a group of datasets by comparing with the corresponding ground truths as well as the classical algorithms. Experimental results show that the proposed method has a higher precision than the compared algorithms in a fully automatic fashion.

This study proposes a local bias field and difference estimation (LBDE) model for medical image segmentation and bias field correction. Firstly, the LBDE model uses a linear combination of a given set of smooth orthogonal basis functions, which is called Chebyshev polynomial, to estimate the bias field. Then, a clustering criterion function is defined by considering the difference between the measured image and approximated image in a small region. By applying this difference in the local region, the LBDE model can obtain accurate segmentation results and estimation of the bias field. Finally, the energy functional is incorporated into a level set formulation with a regularisation term, and it is minimised via the level set evolution process. The LBDE model first appears as a two-phase model and then extends to the multi-phase one. Extensive experiments on medical images demonstrate that the LBDE model achieves more precise segmentation results in terms of Jaccard similarity and dice similarity coefficient than the comparative models. Therefore the proposed model can increase the segmentation accuracy and robustness to noise.

This study proposes a clustering-based colour image segmentation approach consisting of a novel initialisation technique. Colour image segmentation transforms image pixels into regions and a prerequisite for image analysis and computer vision applications. Therefore, colour image segmentation is considered one of the most important processes in image understanding and pattern recognition. This study presents an efficient and adaptive unsupervised approach based on bottom-up red–green–blue (RGB) colour histogram search approach to achieve colour image segmentation. Firstly, the RGB histogram is processed through a double-scan procedure to determine significant modes in each histogram. In the next step, each mode is processed through a bottom-up histogram search approach, completing RGB triplet. The RGB triplets are utilised as the cluster centroids, clustering the pixels into regions and producing the final segmented image. The authors proposed method was compared with several other unsupervised image segmentation algorithms with an extensive experiment performed on various image segmentation evaluation benchmarks. Experimental results show that the proposed algorithm outperforms state-of-the-art algorithms both in terms of features integrity and execution speed.

In this study, the authors propose a novel technique for medical image watermark detection using the concept of the fractional differentiator (FD). The feature of FD as a non-linear high-pass filter helps in watermark detection. In the region of non-interest, the watermark image has been added in a mid-band frequency range of the discrete cosine transform coefficients of different blocks by generating direct spread spectrum sequence. Their scheme produces noise-free watermarked medical images. Furthermore, they derive the test statistics of the proposed detector, which is characterised by the fractional order q. The average errors in pixels (PEs), peak-signal-to-noise ratio (PSNR), structural similarity index (SSIM) and cross-correlation coefficient (CC) have been used to quantify the capability of the proposed technique over some state-of-the-art techniques. The proposed technique shows that at a particular value of fractional order q, there is a significant reduction in average PEs. It causes an increment in PSNR, SSIM and CC. The proposed technique is tested on a large number of medical images and it is found that their proposed technique works better or comparable with other state-of-the-art techniques.

Previously visible watermark removal algorithms required the location of known watermarks. A corresponding removal algorithm is then proposed based on the location and the feature of the watermark. If the location of the watermark is random or the watermark has different angles, the watermark removal algorithm will encounter problems. The authors recommend a visible watermark removal algorithm based on generative adversarial networks (GANs) and self-attention mechanisms. During the training, the authors introduce a GANs model to build mappings between watermarked images and real images. The authors observe that the feature of the watermarked region in different watermarked images is invariant in nature, and the other regions are changed. The self-attention layer will automatically focus on this invariant feature. Experiments on two public datasets prove that the authors’ model has gained excellent performance. Compared with the other four most competitive watermark removal models, the authors improve the watermark removal rate indicator from 17 to 92%. For the other four evaluation indicators, the authors have improved performance by up to 20%.

Application-specific data for certain biometric applications are often not sufficiently available. The authors present a solution for face recognition with limited application-specific data. Existing methods often use a classifier with convolutional neural networks (CNNs) as feature extractors. The CNNs are trained with massive general (i.e. not application specific) data and the classifier is trained with application-specific data. Alternatively, the authors propose a combined training strategy to train the classifier on a balanced mixture of general and application-specific data, such that the recognition performance is maximised. The proposed method largely alleviates the needs for application-specific data. To prove its effectiveness, they apply the proposed method to low-resolution face recognition. Specifically, they use the heterogeneous joint Bayesian (HJB) classifier that is capable of comparing features from the same modality but with different characteristics. To further boost performance, the authors augment the training data by pre-processing it to resemble application-specific data. They conducted extensive experiments on challenging datasets, namely, SCface and COX. The results show that the proposed method improves the true match rate on SCface at a false match rate of 10% by ∼11% and the true match rate on COX at a false match rate of 1% by ∼12%.

A novel single image rain removal model is proposed. Based on the gradient magnitude and direction of rain streaks, the pure rain region can be extracted from the high-frequency component of rainy image. To ensure proper rain removal, a pure rain dictionary is learned from the extracted pure rain region, and the learned rain dictionary is used to reconstruct the rainy mask from the decomposed high-frequency component. To adaptively remove rain pixels and preserve more details of non-rain pixels, a rainy mask is incorporated into the model. In the proposed model, an improved bilateral filter is only used to handle rain pixels. The experimental results show that the proposed model is superior to existing models in resolving the problems of over-smoothing and rain-streak remains in synthetic and real-world rainy images. Consequently, a better quantitative index and visual quality can be achieved.

The accurate segmentation of the optic disc (OD) is important in diagnosing and evaluating many retinal diseases. However, the OD boundary is unclear, making the task of automatic OD segmentation very challenging. Recently,many researchers have applied convolutional neural network (CNN) technology to the automatic segmentation of OD ,and the network has been widened and deepened. It can effectively improves the accuracy of segmentation but also requires high computational complexity and large memory consumption. To overcome the above defects, we propose a segmentation framework of a lightweight cascade CNN. It consists of a designed-to-be-lightweight segmentation network and a shape-refinement network cascade, cascading a shape-refined network behind a segmentation network to compensate for the degraded performance of the segmentation network after lightweight design. We tested our framework on three databases, DRIVE, DIARETDB1, and DRIONS-DB, and found that its segmentation performance is slightly better than that of u-net, and the trainable parameters are approximately 1/35 that of u-net. After verified by the DRIVE dataset, the memory used for training and testing is only about 1/3 of u-net. The method proposed in this paper can greatly reduce trainable parameters and computational resource consumption while guaranteeing satisfactory segmentation performance of the model.