http://iet.metastore.ingenta.com
1887

Reinforcement learning control of a single-link flexible robotic manipulator

Reinforcement learning control of a single-link flexible robotic manipulator

For access to this article, please select a purchase option:

Buy article PDF
$19.95
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Control Theory & Applications — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

In this study, the authors focus on the reinforcement learning control of a single-link flexible manipulator and attempt to suppress the vibration due to its flexibility and lightweight structure. The assumed mode method and the Lagrange's equation are adopted in modelling to enhance the satisfaction of precision. Two radial basis function neural networks (NNs) are employed in the designed control algorithm, actor NN for generating a policy and critic NN for evaluating the cost-to-go. Rigorous stability of the system has been proven via Lyapunov's direct method. Through Matlab simulation and experiment on the Quanser flexible link platform, the superiority and feasibility of the reinforcement learning control are verified.

References

    1. 1)
      • 1. Kiang, C.T., Spowage, A., Yoong, C.K.: ‘Review of control and sensor system of flexible manipulator’, J. Intell. Robot. Syst., 2015, 77, (1), pp. 187213.
    2. 2)
      • 2. Saad, M.: ‘Modeling of a one flexible link manipulator’ (INTECH Open Access Publisher, 2010).
    3. 3)
      • 3. Korayem, M.H., Rahimi, H., Nikoobin, A.: ‘Mathematical modeling and trajectory planning of mobile manipulators with flexible links and joints’, Appl. Math. Model., 2012, 36, (7), pp. 32293244.
    4. 4)
      • 4. Sun, C., He, W., Hong, J.: ‘Neural network control of a flexible robotic manipulator using the lumped spring-mass mode’, IEEE Trans. Syst. Man Cybern., Syst., 2017, in press, DOI: 101109/TSMC20162562506.
    5. 5)
      • 5. Liu, Y.J., Tong, S.C.: ‘Barrier Lyapunov functions for Nussbaum gain adaptive control of full state constrained nonlinear systems’, Automatica, 2017, 76, pp. 143152.
    6. 6)
      • 6. Li, H., Shi, P., Yao, D.: ‘Adaptive sliding mode control of Markov jump nonlinear systems with actuator faults’, IEEE Trans. Autom. Control, 2016, 61, (9), pp. 27452751.
    7. 7)
      • 7. Li, Y., Ge, S.S.: ‘Impedance learning for robots interacting with unknown environments’, IEEE Trans. Control Syst. Technol., 2014, 22, (4), pp. 14221432.
    8. 8)
      • 8. He, W., Ge, W., Li, Y., et al: ‘Model Identification and Control Design for a Humanoid Robot’, IEEE Trans. Syst. Man Cybern., Syst., 2017, 47, (1), pp. 4557.
    9. 9)
      • 9. Kang, Y., Zhai, D.H., Liu, G.P., et al: ‘Stability analysis of a class of hybrid stochastic retarded systems under asynchronous switching’, IEEE Trans. Autom. Control, 2014, 59, (6), pp. 15111523.
    10. 10)
      • 10. Kang, Y., Zhai, D.H., Liu, G.P., et al: ‘On input-to-state stability of switched stochastic nonlinear systems under extended asynchronous switching’, IEEE Trans. Cybern., 2016, 46, (5), pp. 10921105.
    11. 11)
      • 11. Wang, J.W., Wu, H.N., Li, H.X.: ‘Stochastically exponential stability and stabilization of uncertain linear hyperbolic PDE systems with Markov jumping parameters’, Automatica, 2012, 48, (3), pp. 569576.
    12. 12)
      • 12. Zhang, S., He, X., Yang, C.: ‘Vibration control of a flexible marine with joint angle constraint’, Nonlinear Dyn., 2017, 87, (1), pp. 617632.
    13. 13)
      • 13. He, W., Ge, S.S., Huang, D.: ‘Modeling and vibration control for a nonlinear moving string with output constraint’, IEEE/ASME Trans. Mechatronics, 2015, 20, (4), pp. 18861897.
    14. 14)
      • 14. He, W., Ge, S.S., Zhang, S.: ‘Adaptive Boundary Control of a Nonlinear Flexible String System’, IEEE Transactions on Control Systems Technology, 2014, 22, (3), pp. 10881093.
    15. 15)
      • 15. He, W., Nie, S., Meng, T., et al: ‘Modeling and Vibration Control for a Moving Beam with Application in a Drilling Riser’, IEEE Trans. Control Syst. Technol., 2017, 25, (3), pp. 10361043.
    16. 16)
      • 16. Zhang, S., Huang, D.: ‘End-point regulation and vibration suppression of a flexible robotic manipulator’, Asian J. Control, 2017, 19, (1), pp. 245254.
    17. 17)
      • 17. Wu, H.N., Wang, J.W.: ‘Static output feedback control via PDE boundary and ODE measurements in linear cascaded ODE–beam systems’, Automatica, 2014, 50, (11), pp. 27872798.
    18. 18)
      • 18. Cheong, J., Lee, S.: ‘Linear pid composite controller and its tuning for flexible link robots’, J. Vib. Control, 2008, 14, (3), pp. 291318.
    19. 19)
      • 19. Zhang, L., Liu, J.: ‘Observer-based partial differential equation boundary control for a flexible two-link manipulator in task space’, IET Control Theory Appl., 2012, 6, (13), pp. 21202133.
    20. 20)
      • 20. He, W., Meng, T.: ‘Adaptive Control of a Flexible String System with Input Hysteresis’, IEEE Trans. Control Syst Technol., 2017, in press, DOI: 10.1109/TCST.2017.2669158.
    21. 21)
      • 21. Liu, Z., Liu, J.K., He, W.: ‘Modeling and vibration control of a flexible aerial refueling hose with variable lengths and input constraint’, Automatica, 2017, 77, (3), pp. 302310.
    22. 22)
      • 22. He, W., Meng, T., Huang, D., et al: ‘Adaptive Boundary Iterative Learning Control for an Euler-Bernoulli Beam System with Input Constraint’, IEEE Trans. Neural Netw. Learn. Syst, 2017, in press, DOI: 10.1109/TNNLS.2017.2673865.
    23. 23)
      • 23. Zhang, S., He, W., Huang, D.: ‘Active vibration control for a flexible string system with input backlash’, IET Control Theory Appl., 2016, 10, (7), pp. 800805.
    24. 24)
      • 24. Ulrich, S., Sasiadek, J.Z., Barkana, I.: ‘Modeling and direct adaptive control of a flexible-joint manipulator’, J. Guid. Control Dyn., 2012, 35, (1), pp. 2539.
    25. 25)
      • 25. Corradini, M.L., Fossi, V., Giantomassi, A., et al: ‘Discrete time sliding mode control of robotic manipulators: development and experimental validation’, Control Eng. Pract., 2012, 20, (8), pp. 816822.
    26. 26)
      • 26. Mahmood, I.A., Moheimani, S.O.R., Bhikkaji, B.: ‘Precise tip positioning of a flexible manipulator using resonant control’, IEEE/ASME Trans. Mechatronics, 2008, 13, (2), pp. 180186.
    27. 27)
      • 27. Pereira, E., Aphale, S.S., Feliu, V., et al: ‘Integral resonant control for vibration damping and precise tip-positioning of a single-link flexible manipulator’, IEEE/ASME Trans. Mechatronics, 2011, 16, (2), pp. 232240.
    28. 28)
      • 28. Yang, C., Jiang, Y., Li, Z., et al: ‘Neural control of bimanual robots with guaranteed global stability and motion precision’, IEEE Trans. Ind. Inf., 2017, in press, DOI: 10.1109/TII.2016.2612646.
    29. 29)
      • 29. He, W., Ge, S.S., Li, Y., et al: ‘Neural network control of a rehabilitation robot by state and output feedback’, J. Intell. Robot. Syst., 2015, 80, (1), pp. 117.
    30. 30)
      • 30. Kumar, N., Panwar, V., Sukavanam, N., et al: ‘Neural network based hybrid force/position control for robot manipulators’, Int. J. Prec. Eng. Manuf., 2011, 12, (3), pp. 419426.
    31. 31)
      • 31. Yang, C., Li, Z., Cui, R., et al: ‘Neural network-based motion control of an underactuated wheeled inverted pendulum model’, IEEE Trans. Neural Netw. Learn. Syst., 2014, 25, (11), pp. 20042016.
    32. 32)
      • 32. Qiao, H., Peng, J.G., Xu, Z.B., et al: ‘A reference model approach to stability analysis of neural networks’, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2003, 33, (6), pp. 925936.
    33. 33)
      • 33. He, W., Ouyang, Y., Hong, J.: ‘Vibration control of a flexible robotic manipulator in the presence of input deadzone’, IEEE Trans. Ind. Inf., 2017, 13, (1), pp. 4859.
    34. 34)
      • 34. Tang, Y., Sun, F., Sun, Z.: ‘Neural network control of flexible-link manipulators using sliding mode’, Neurocomputing, 2006, 70, (1), pp. 288295.
    35. 35)
      • 35. Zhou, Q., Shi, P., Tian, Y., et al: ‘Approximation-based adaptive tracking control for MIMO nonlinear systems with input saturation’, IEEE Trans. Cybern., 2015, 45, (10), pp. 21192128.
    36. 36)
      • 36. Li, Z., Huang, Z., He, W., et al: ‘Adaptive impedance control for an upper limb robotic exoskeleton using biological signals’, IEEE Trans. Ind. Electron., 2017, 64, (2), pp. 16641674.
    37. 37)
      • 37. Tian, L., Collins, C.: ‘Adaptive neuro-fuzzy control of a flexible manipulator’, IEEE/ASME Trans. Mechatronics, 2005, 15, (10), pp. 13051320.
    38. 38)
      • 38. Kaelbling, L.P., Littman, M.L., Moore, A.W.: ‘Reinforcement learning: a survey’, J. Artif. Intell. Res., 1996, 4, (1), pp. 237285.
    39. 39)
      • 39. He, P., Jagannathan, S.: ‘Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints’, IEEE Trans. Syst. Man Cybern. B, Cybern., 2007, 37, (2), pp. 425436.
    40. 40)
      • 40. Lin, C.K.: ‘H reinforcement learning control of robot manipulators using fuzzy wavelet networks’, Fuzzy Sets Syst., 2009, 160, (12), pp. 17651786.
    41. 41)
      • 41. Busoniu, L., Babuska, R., De Schutter, B., et al: ‘Reinforcement learning and dynamic programming using function approximators’ (CRC Press, 2010) vol. 39.
    42. 42)
      • 42. Xu, B., Yang, C., Shi, Z.: ‘Reinforcement learning output feedback NN control using deterministic learning technique’, IEEE Trans. Neural Netw. Learn. Syst., 2014, 25, (3), pp. 635641.
    43. 43)
      • 43. Liu, Y.J., Tang, L., Tong, S., et al: ‘Reinforcement learning design-based adaptive tracking control with less learning parameters for nonlinear discrete-time MIMO systems’, IEEE Trans. Neural Netw. Learn. Syst., 2015, 26, (1), pp. 165176.
    44. 44)
      • 44. Vamvoudakis, K.G., Lewis, F.L.: ‘Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem’, Automatica, 2010, 46, (5), pp. 878888.
    45. 45)
      • 45. Lewis, F.L., Vrabie, D.: ‘Reinforcement learning and adaptive dynamic programming for feedback control’, IEEE Circuits Syst. Mag., 2009, 9, (3), pp. 3250.
    46. 46)
      • 46. Lewis, F.L., Vrabie, D., Vamvoudakis, K.G.: ‘Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers’, IEEE Control Syst., 2012, 32, (6), pp. 76105.
    47. 47)
      • 47. Tang, L., Liu, Y.J., Tong, S.: ‘Adaptive neural control using reinforcement learning for a class of robot manipulator’, Neural Comput. Appl., 2014, 25, (1), pp. 135141.
    48. 48)
      • 48. Clough, R., Penzien, J.: ‘Dynamics of structures’ (McGraw-Hill, 1975).
    49. 49)
      • 49. He, W., Dong, Y., Sun, C.: ‘Adaptive neural impedance control of a robotic manipulator with input saturation’, IEEE Trans. Syst. Man Cybern., Syst., 2016, 46, (3), pp. 334344.
    50. 50)
      • 50. Vrabie, D., Pastravanu, O., Abu-Khalaf, M., et al: ‘Adaptive optimal control for continuous-time linear systems based on policy iteration’, Automatica, 2009, 45, (2), pp. 477484.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cta.2016.1540
Loading

Related content

content/journals/10.1049/iet-cta.2016.1540
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address