Orthogonality-based label coding is an often-used technique in multi-class classification. Through coding the labels into some multi-dimensional orthogonal codewords, many binary classifiers can be naturally extended to multi-class cases. For an unseen sample, the classifiers firstly estimate its codeword and then compute the corresponding distances from the labels. Finally, the nearest one is assigned as its class label. However, these classifiers actually hardly guarantee that the estimated codewords still maintain the inter-orthogonality with the other classes, which more likely causes the codewords in different classes overlapping each other to some extent and thus affects the classification performance. Proposed is a novel label correction strategy which aims to keep as much as possible orthogonality between the estimated sample codewords and the other classes’ labels in order to preserve further as much as possible the inter-orthogonality of the codewords. The strategy is combined with two state-of-the-art classifiers: regularised least square classifier and the least square support vector machine. Experiments on UCI datasets demonstrate the effectiveness of the method.