Phenomenological appearance models capture surface appearance through mathematical modelling of the reflection process. Theoretically, the space of all possible images of a fixed-pose object under all possible illumination conditions is infinite dimensional. Nonetheless, because of their low-frequency nature, irradiance signals can be represented using low-order basis functions. Discounting subsurface scattering and surface emittance, this work seeks to address the question; how to compactly and accurately represent image irradiance under unknown general illumination, given that a surface point sees its surrounding world through the local upper hemisphere oriented by the surface normal at this point. In this study, we formulate the image formation process of isotropic surface reflectance under arbitrary distant illumination in the frequency space while addressing the physical compliance of hemispherical basis for representing surface reflectance, for example, Helmholtz reciprocity and isotropy. The term ‘irradiance harmonics’ is also defined which enables decoupling illumination and reflectance from the underlying geometry and pose. We provide a closed form of the energy content being maintained by different reflectance modes of the proposed irradiance harmonics. Since specular materials tend to require more basis functions when compared with diffuse ones, the presented harmonics captures same cumulative energy content, by providing larger number of orthogonal irradiance basis, at lower illumination orders when compared to similar basis in literature.

Owing to losing the detailed information, the low-resolution problem in face recognition degrades the recognition performance dramatically. To overcome this problem, a novel face-recognition system has been proposed, consisting of the extracted feature vectors from the multiple-size discrete cosine transforms (mDCTs) and the recognition mechanism with selective Gaussian mixture models (sGMMs). The mDCT could extract enough visual features from low-resolution face images while the sGMM could exclude unreliable observation features in recognition phase. Thus, the mDCT and the sGMM can greatly improve recognition rate at low-resolution conditions. Experiments are carried out on George Tech and AR facial databases in 16 × 16 and 12 × 12 pixels resolution. The results show that the proposed system achieves better performance than the existing methods for low-resolution face recognition.

Leaf area measurements are commonly obtained by destructive and laborious practice. This study shows how stereo and time-of-flight (ToF) images can be combined for non-destructive automatic leaf area measurements. The authors focus on some challenging plant images captured in a greenhouse environment, and show that even the state-of-the-art stereo methods produce unsatisfactory results. By transforming depth information in a ToF image to a localised search range for dense stereo, a global optimisation strategy is adopted for producing smooth results that preserve discontinuity. They also use edges of colour and disparity images for automatic leaf detection and develop a smoothing method necessary for accurately estimating surface area. In addition to show that combining stereo and ToF images gives superior qualitative and quantitative results, 149 automatic measurements on leaf area using the authors system in a validation trial have a correlation of 0.97 with true values and the root-mean-square error is 10.97 cm², which is 9.3% of the average leaf area. Their approach could potentially be applied for combining other modalities of images with large difference in image resolutions and camera positions.

Precise knowledge of the 3D shape of clinical crowns is crucial for the treatment of malocclusion problems as well as several endodontic procedures. While computed tomography would present such information, it is believed there is no threshold radiation dose below which it is considered safe. In this study, the authors propose an appearance based approach which allows for the reconstruction of plausible human jaw 3D models given a single optical image with unknown illumination. Appearance bases are analytically constructed using the frequency-based representation of the irradiance equation while incorporating prior information about natural illumination and teeth reflectance. The inherent relation between the photometric information and the underlying 3D shape is formulated as a statistical model where the coupled effect of illumination and reflectance is modelled using the Helmholtz hemispherical harmonics-based irradiance harmonics whereas the principle component regression is deployed to carry out the estimation of 3D shapes. The authors' approach relaxes limiting assumptions of conventional shape-from-shading approaches while being able to reconstruct tooth occlusal surface with challenging conditions, such as scattered specular spots and significant changes in colour and albedo characteristics resulting from tooth filling. Vis-à-vis dental applications, the results demonstrate a significant increase in accuracy in favour of the proposed approach.

Visual tracking is a popular and challenging topic in computer vision and robotics. Owing to changes in the appearance of the target and complicated variations that may occur in various scenes, online learning scheme is necessary for advanced visual tracking framework to adopt. This paper briefly introduces the challenges and applications of visual tracking and focuses on discussing the state-of-the-art online-learning-based tracking methods by category. We provide detail descriptions of representative methods in each category, and examine their pros and cons. Moreover, several most representative algorithms are implemented to provide quantitative reference. At last, we outline several trends for future visual tracking research.

Facial expression recognition (FER) plays an important role in human–computer interaction. The recent years have witnessed an increasing trend of various approaches for the FER, but these approaches usually do not consider the effect of individual differences to the recognition result. When the face images change from neutral to a certain expression, the changing information constituted of the structural characteristics and the texture information can provide rich important clues not seen in either face image. Therefore it is believed to be of great importance for machine vision. This study proposes a novel FER algorithm by exploiting the structural characteristics and the texture information hiding in the image space. Firstly, the feature points are marked by an active appearance model. Secondly, three facial features, which are feature point distance ratio coefficient, connection angle ratio coefficient and skin deformation energy parameter, are proposed to eliminate the differences among the individuals. Finally, a radial basis function neural network is utilised as the classifier for the FER. Extensive experimental results on the Cohn–Kanade database and the Beihang University (BHU) facial expression database show the significant advantages of the proposed method over the existing ones.

Example-based statistical face models using principle component analysis (PCA) have been widely deployed for three-dimensional (3D) face reconstruction and face recognition. The two common factors that are generally concerned with such models are the size of the training dataset and the selection of different examples in the training set. The representational power (RP) of an example-based model is its capability to depict a new 3D face for a given 2D face image. The RP of the model can be increased by correspondingly increasing the number of training samples. In this contribution, a novel approach is proposed to increase the RP of the 3D face reconstruction model by deforming a set of examples in the training dataset. A PCA-based 3D face model is adapted for each new near frontal input face image to reconstruct the 3D face shape. Further an extended Tikhonov regularisation method has been employed to reconstruct 3D face shapes from a set of facial points. The results justify that the proposed adaptive PCA-based model considerably improves the RP of the standard PCA-based model and outperforms it with a 95% confidence level.