We are concerned with a kind of iterative method for computing the Moore-Penrose inverse, which can be considered as a discrete-time form of recurrent neural networks. We study the momentum learning scheme of the method and discuss its semi-convergence when computing the Moore-Penrose inverse of a rankdeficient matrix. We prove the semi-convergence for our new acceleration algorithm and obtain the optimal momentum factor which makes the fastest semi-convergence. Numerical tests demonstrate the effectiveness of our new acceleration algorithm.