A new adaptive algorithm for the robust estimation of parameters of linear dynamic discrete-time systems in the presence of non-Gaussian impulsive noise within a measurement sequence is proposed in this study. Starting from the theory of robust estimation, a simple, adaptive, practically applicable robust approximate maximum likelihood algorithm is derived that, in the cases of contaminated normal distribution of measurement noise, demonstrates a high level of efficiency. The QQ-plot technique, combined with data cleaning based on the robustified winsorisation technique, is used as a framework for the classification of sorted data into the class of regular normally distributed data and the class of irregular data belonging to the contaminating distribution with a variance that is much greater than nominal. The link between the QQ-plot technique and a specific linear regression is established, so that the estimation of statistical parameters of the contaminated measurement distribution is performed using the least-squares technique. Then, the suboptimal maximum likelihood criterion is defined, and the system parameter estimation problem is solved robustly, using the proposed recursive robust parameter estimation scheme. Simulation results illustrate the discussion and show the efficiency of the proposed adaptive recursive parameter estimation algorithm in the presence of glint spikes or outliers.