Human Epithelial type 2 (HEp-2) cells are the most common substrates for anti-nuclear antibodies detection. Traditional manual diagnosis heavily depends on the experience of histopathologists, which is time consuming and subject to subjective mistakes. With the recent progress of digital scanners and dramatic development in computer vision techniques, computer-aided diagnosis has now become achievable. In this paper a novel automatic system is proposed to classify the HEp2 cell images into six categories. Along with a set of local gradient based textural descriptors, we introduce a novel objectbased method to decompose the binary image into primitive objects and represent them with a set of morphological features. Random forest is then applied for classification. The advantages of this system are as following: (1) robustness against the changes of intensity and rotation, (2) more discriminative information compared to normal morphological descriptors. We evaluate the proposed approach using the publicly available ICPR 2012 datasets. The experimental results show that the proposed method achieves comparable performance with the state-of-the-art methods.