A new method for applying orthogonal transforms in signal bias removal (SBR) for adverse Mandarin speech recognition (MSR) is proposed. The orthogonal transform process is performed in a moving window manner to extract features from the input speech. Codewords are then obtained by matching high-order, bias-free features with pre-trained codebooks for bias estimation. The effectiveness of the method has been confirmed by an experiment involving multi-speaker adverse continuous MSR. Significant improvements in the recognition accuracy and computation time were achieved as compared with the conventional SBR method.