0
Research Papers

Online Policy Iteration-Based Tracking Control of Four Wheeled Omni-Directional Robots

[+] Author and Article Information
Arash Sheikhlar

Faculty of Electrical,
Biomedical and Mechatronics Engineering
Qazvin Branch,
Islamic Azad University,
Qazvin 1478735564, Iran
e-mail: arash.sheikhlar@qiau.ac.ir

Ahmad Fakharian

Faculty of Electrical,
Biomedical and Mechatronics Engineering
Qazvin Branch,
Islamic Azad University,
Qazvin 1478735564, Iran
e-mail: ahmad.fakharian@qiau.ac.ir

1Present address: Young Researchers and Elite Club, Qazvin Branch, Islamic Azad University, Qazvin, Iran.

2Corresponding author.

Contributed by the Dynamic Systems Division of ASME for publication in the JOURNAL OF DYNAMIC SYSTEMS, MEASUREMENT,AND CONTROL. Manuscript received February 19, 2017; final manuscript received January 27, 2018; published online March 28, 2018. Assoc. Editor: Yongchun Fang.

J. Dyn. Sys., Meas., Control 140(8), 081017 (Mar 28, 2018) (12 pages) Paper No: DS-17-1108; doi: 10.1115/1.4039287 History: Received February 19, 2017; Revised January 27, 2018

In this paper, online policy iteration reinforcement learning (RL) algorithm is proposed for motion control of four wheeled omni-directional robots. The algorithm solves the linear quadratic tracking (LQT) problem in an online manner using real-time measurement data of the robot. This property enables the tracking controller to compensate the alterations of dynamics of the robot's model and environment. The online policy iteration based tracking method is employed as low level controller. On the other side, a proportional derivative (PD) scheme is performed as supervisory planning system (high level controller). In this study, the followed paths of online and offline policy iteration algorithms are compared in a rectangular trajectory in the presence of slippage drawback and motor heat. Simulation and implementation results of the methods demonstrate the effectiveness of the online algorithm compared to offline one in reducing the command trajectory tracking error and robot's path deviations. Besides, the proposed online controller shows a considerable ability in learning appropriate control policy on different types of surfaces. The novelty of this paper is proposition of a simple-structure learning based adaptive optimal scheme that tracks the desired path, optimizes the energy consumption, and solves the uncertainty problem in omni-directional wheeled robots.

Copyright © 2018 by ASME
Your Session has timed out. Please sign back in to continue.

References

Kim, H. , and Kim, B. K. , 2014, “ Online Minimum-Energy Trajectory Planning and Control on a Straight-Line Path for Three-Wheeled Omnidirectional Mobile Robots,” IEEE Trans. Ind. Electron., 61(9), pp. 4771–4779. [CrossRef]
Jia, Z. J. , Song, Y. D. , and Cai, W. C. , 2013, “ Bio-Inspired Approach for Smooth Motion Control of Wheeled Mobile Robots,” Cognit. Comput., 5(2), pp. 252–263. [CrossRef]
Kanjanawanishkul, K. , 2015, “ Omnidirectional Wheeled Mobile Robots: Wheel Types and Practical Applications,” Int. J. Adv. Mechatronic Syst., 6(6), pp. 289–302. [CrossRef]
Burl, J. B. , 1998, “Linear Optimal Control: H (2) and H (Infinity) Methods,” Addison-Wesley Longman Publishing, Boston, MA.
Sherback, M. , Purwin, O. , and D'Andrea, R. , 2006, “ Real-Time Motion Planning and Control in the 2005 Cornell Robocup System,” Rob. Motion Control, 335, pp. 245–263. [CrossRef]
Bruce, J. R. , 2006, “ Real-Time Motion Planning and Safe Navigation in Dynamic Multi-Robot Environments,” Ph.D. dissertation, Carnegie Mellon University, Washington, DC. http://reports-archive.adm.cs.cmu.edu/anon/2006/CMU-CS-06-181.pdf
Purwin, O. , and D'Andrea, R. , 2006, “ Trajectory Generation and Control for Four Wheeled Omnidirectional Vehicles,” Rob. Auton. Syst., 54(1), pp. 13–22. [CrossRef]
Juang, C. F. , Lai, M. G. , and Zeng, W. T. , 2015, “ Evolutionary Fuzzy Control and Navigation for Two Wheeled Robots Cooperatively Carrying an Object in Unknown Environments,” IEEE Trans. Cybern., 45(9), pp. 1731–1743. [CrossRef] [PubMed]
Chwa, D. , 2012, “ Fuzzy Adaptive Tracking Control of Wheeled Mobile Robots With State-Dependent Kinematic and Dynamic Disturbances,” IEEE Trans. Fuzzy Syst., 20(3), pp. 587–593. [CrossRef]
Yang, S. X. , Zhu, A. , Yuan, G. , and Meng, M. Q. H. , 2012, “ A Bioinspired Neurodynamics-Based Approach to Tracking Control of Mobile Robots,” IEEE Trans. Ind. Electron., 59(8), pp. 3211–3220. [CrossRef]
Kayacan, E. , Kayacan, E. , Ramon, H. , and Saeys, W. , 2013, “ Adaptive Neuro-Fuzzy Control of a Spherical Rolling Robot Using Sliding-Mode-Control-Theory-Based Online Learning Algorithm,” IEEE Trans. Cybern., 43(1), pp. 170–179. [CrossRef] [PubMed]
Gao, H. , Song, X. , Ding, L. , Xia, K. , Li, N. , and Deng, Z. , 2014, “ Adaptive Motion Control of Wheeled Mobile Robot With Unknown Slippage,” Int. J. Control, 87(8), pp. 1513–1522. [CrossRef]
Hoang, N. B. , and Kang, H. J. , 2016, “ Neural Network-Based Adaptive Tracking Control of Mobile Robots in the Presence of Wheel Slip and External Disturbance Force,” Neurocomputing, 188, pp. 12–22. [CrossRef]
Boukens, M. , and Boukabou, A. , 2017, “ Design of an Intelligent Optimal Neural Network-Based Tracking Controller for Nonholonomic Mobile Robot Systems,” Neurocomputing, 226, pp. 46–57. [CrossRef]
Su, K. H. , Chen, Y. Y. , and Su, S. F. , 2010, “ Design of Neural-Fuzzy-Based Controller for Two Autonomously Driven Wheeled Robot,” Neurocomputing, 73(13–15), pp. 2478–2488. [CrossRef]
Li, S. , Wang, H. , and Rafique, M. U. , 2017, “ A Novel Recurrent Neural Network for Manipulator Control With Improved Noise Tolerance,” IEEE Trans. Neural Networks Learn. Syst., 36(4), pp. 424–440.
Li, S. , Zhang, Y. , and Jin, L. , 2016, “ Kinematic Control of Redundant Manipulators Using Neural Networks,” IEEE Transactions Neural Networks Learning Syst., 28(10), pp. 2243–2254.
Zhang, Y. , and Li, S. , 2017, “ Predictive Suboptimal Consensus of Multiagent Systems With Nonlinear Dynamics,” IEEE Trans. Syst., Man, Cybern.: Syst., 47(7), pp. 1701–1711.
Li, S. , Zhou, M. , Luo, X. , and You, Z. H. , 2017, “ Distributed Winner-Take-All in Dynamic Networks,” IEEE Trans. Autom. Control, 62(2), pp. 577–589. [CrossRef]
Jin, L. , Li, S. , La, H. M. , and Luo, X. , 2017, “ Manipulability Optimization of Redundant Manipulators Using Dynamic Neural Networks,” IEEE Trans. Ind. Electron., 64(6), pp. 4710–4720.
Sheikhlar, A. , Fakharian, A. , Beik-Mohammadi, H. , and Adhami-Mirhosseini, A. , 2016, “ Design and Implementation of Self-Adaptive PD Controller Based on Fuzzy Logic Algorithm for Omni-Directional Fast Robots in Presence of Model Uncertainties,” Int. J. Uncertainty, Fuzziness Knowl.-Based Syst., 24(5), pp. 761–780. [CrossRef]
Sheikhlar, A. , Zarghami, M. , Fakharian, A. , and Menhaj, M. B. , 2013, “ Delay Compensation on Fuzzy Trajectory Tracking Control of Omni-Directional Mobile Robots,” Amirkabir Int. J. Electr. Electron. Eng., 45(2), pp. 57–64.
Sharbafi, M. A. , Lucas, C. , and Daneshvar, R. , 2010, “ Motion Control of Omni-Directional Three-Wheel Robots by Brain-Emotional-Learning-Based Intelligent Controller,” IEEE Trans. Syst., Man, Cybern., Part C (Appl. Rev.), 40(6), pp. 630–638. [CrossRef]
Kiumarsi-Khomartash, B. , Lewis, F. L. , Naghibi-Sistani, M. B. , and Karimpour, A. , 2013, “ Optimal Tracking Control for Linear Discrete-Time Systems Using Reinforcement Learning,” IEEE 52nd Annual Conference on Decision and Control (CDC), Florence, Italy, Dec. 10–13, pp. 3845–3850.
Hashemi, E. , Jadidi, M. G. , and Jadidi, N. G. , 2011, “ Model-Based PI–Fuzzy Control of Four-Wheeled Omni-Directional Mobile Robots,” Rob. Auton. Syst., 59(11), pp. 930–942. [CrossRef]
Ben-Israel, A. , and Greville, T. N. , 2003, “Generalized Inverses: Theory and Applications, Vol. 15, Springer Science & Business Media, New York.
Montone, D., 2017, “Temperature Effects on Motor Performance,” AMETEK Advanced Motion Solutions, Berwyn, PA, accessed Feb. 21, 2018, http://www.haydonkerkpittman.com/-/media/ametekhaydonkerk/downloads/white-papers/temperature_effects_on_dc_motor_performance_1.pdf?la=en
Kiumarsi, B. , Lewis, F. L. , Modares, H. , Karimpour, A. , and Naghibi-Sistani, M. B. , 2014, “ Reinforcement Q-Learning for Optimal Tracking Control of Linear Discrete-Time Systems With Unknown Dynamics,” Automatica, 50(4), pp. 1167–1175. [CrossRef]
Hewer, G. , 1971, “ An Iterative Technique for the Computation of the Steady-State Gains for the Discrete Optimal Regulator,” IEEE Trans. Autom. Control, 16(4), pp. 382–384. [CrossRef]
Liu, Y. , Zhu, J. J. , Williams, R. L. , and Wu, J. , 2008, “ Omni-Directional Mobile Robot Controller Based on Trajectory Linearization,” Rob. Auton. Syst., 56(5), pp. 461–479. [CrossRef]
Zhao, W. , Kim, B. H. , Larson, A. C. , and Voyles, R. M. , 2005, “ FPGA Implementation of Closed-Loop Control System for Small-Scale Robot,” 12th International Conference on Advanced Robotics (ICAR'05), Seattle, WA, July 18–20, pp. 70–77.
Chaiso, K. , and Sukvichai, K. , 2011, “Skuba 2011 Extended Team Description,” RoboCup Wiki as extended team description of SKUBA team, Istanbul, Turkey, accessed Feb. 21, 2018, http://wiki.robocup.org/images/4/4c/Small_Size_League_-_RoboCup_2011_-_ETDP_Skuba.pdf

Figures

Grahic Jump Location
Fig. 1

Geometry and frames of a four wheeled robot

Grahic Jump Location
Fig. 2

Closed-loop scheme of omni-directional wheeled robot

Grahic Jump Location
Fig. 3

Angular velocity of the first wheel and related control signal

Grahic Jump Location
Fig. 4

Angular velocity of the second wheel and related control signal

Grahic Jump Location
Fig. 5

Angular velocity of the third wheel and related control signal

Grahic Jump Location
Fig. 6

Angular velocity of the fourth wheel and related control signal

Grahic Jump Location
Fig. 7

Position and orientation of the robot in rectangular path tracking

Grahic Jump Location
Fig. 8

Rectangular path tracking

Grahic Jump Location
Fig. 9

Tracking error of position and orientation of the robot in the rectangular path tracking

Grahic Jump Location
Fig. 10

Overall structure of data flow in SSL soccer robots

Grahic Jump Location
Fig. 11

The MRL SSL robot that is used for experiments

Grahic Jump Location
Fig. 12

The MRL electronic mainboard that is mounted on the robot

Grahic Jump Location
Fig. 13

Angular velocity of the first wheel

Grahic Jump Location
Fig. 14

Angular velocity of the second wheel

Grahic Jump Location
Fig. 15

Angular velocity of the third wheel

Grahic Jump Location
Fig. 16

Angular velocity of the fourth wheel

Grahic Jump Location
Fig. 17

Rectangular path tracking

Grahic Jump Location
Fig. 18

Tracking error of position and orientation of the robot in the rectangular path tracking

Grahic Jump Location
Fig. 19

Classic SSL low-level controller

Grahic Jump Location
Fig. 20

Target point tracking of the robot in ordinary condition. The start and target points are indicated by big and small circles, respectively.

Grahic Jump Location
Fig. 21

Target point tracking of the robot after changing the surface's carpet. The start and target points are indicated by big and small circles, respectively.

Grahic Jump Location
Fig. 22

Angular velocity of the first wheel after changing the surface's carpet

Grahic Jump Location
Fig. 23

Angular velocity of the second wheel after changing the surface's carpet

Grahic Jump Location
Fig. 24

Angular velocity of the third wheel after changing the surface's carpet

Grahic Jump Location
Fig. 25

Angular velocity of the fourth wheel after changing the surface's carpet

Grahic Jump Location
Fig. 26

Convergence of some of P elements to their optimum values in online policy iteration

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In