0
Journal of Dynamic Systems, Measurement, and Control | Volume 129 | Issue 5 | TECHNICAL PAPERS
TECHNICAL PAPERS

Decentralized Coordinated Motion Control of Two Hydraulic Actuators Handling a Common Object

[+] Author and Article Information
Mark Karpenko

Department of Mechanical and Manufacturing Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6, Canada

Nariman Sepehri1

Department of Mechanical and Manufacturing Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6, Canadanariman@cc.umanitoba.ca

John Anderson

Department of Computer Science, University of Manitoba, Winnipeg, Manitoba, R3T 5V6, Canada

1

Corresponding author.

J. Dyn. Sys., Meas., Control 129(5), 729-741 (Jan 25, 2007) (13 pages) doi:10.1115/1.2764516 History: Received June 23, 2006; Revised January 25, 2007
Article
References
Figures
Tables
Citing Articles

In this paper, reinforcement learning is applied to coordinate, in a decentralized fashion, the motions of a pair of hydraulic actuators whose task is to firmly hold and move an object along a specified trajectory under conventional position control. The learning goal is to reduce the interaction forces acting on the object that arise due to inevitable positioning errors resulting from the imperfect closed-loop actuator dynamics. Each actuator is therefore outfitted with a reinforcement learning neural network that modifies a centrally planned formation constrained position trajectory in response to the locally measured interaction force. It is shown that the actuators, which form a multiagent learning system, can learn decentralized control strategies that reduce the object interaction forces and thus greatly improve their coordination on the manipulation task. However, the problem of credit assignment, a common difficulty in multiagent learning systems, prevents the actuators from learning control strategies where each actuator contributes equally to reducing the interaction force. This problem is resolved in this paper via the periodic communication of limited local state information between the reinforcement learning actuators. Using both simulations and experiments, this paper examines some of the issues pertaining to learning in dynamic multiagent environments and establishes reinforcement learning as a potential technique for coordinating several nonlinear hydraulic manipulators performing a common task.
Copyright © 2007 by American Society of Mechanical Engineers
Your Session has timed out. Please sign back in to continue.

References

Figures

Grahic Jump Location
+Schematic of a typical hydraulic actuator for mathematical modeling
View Large  |  Save Figure  |  Download Slide (.ppt)
Schematic of a typical hydraulic actuator for mathematical modeling
Figure 1

Schematic of a typical hydraulic actuator for mathematical modeling

Grahic Jump Location
+Schematic of coordinated positioning task: (a) definition of reference trajectories x1d and x2d with respect to desired object position xd; (b) deformation of object due to relative actuator positioning error
View Large  |  Save Figure  |  Download Slide (.ppt)
Schematic of coordinated positioning task: (a) definition of reference trajectories x1d and x2d with respect to desired object position xd; (b) deformation of object due to relative actuator positioning error
Figure 2

Schematic of coordinated positioning task: (a) definition of reference trajectories x1d and x2d with respect to desired object position xd; (b) deformation of object due to relative actuator positioning error

Grahic Jump Location
+Schematic of coordinated positioning system: (a) decentralized control scheme using localized position and RLNN controllers; (b) architecture of local reinforcement learning control system showing signals used for learning as dashed lines
View Large  |  Save Figure  |  Download Slide (.ppt)
Schematic of coordinated positioning system: (a) decentralized control scheme using localized position and RLNN controllers; (b) architecture of local reinforcement learning control system showing signals used for learning as dashed lines
Figure 3

Schematic of coordinated positioning system: (a) decentralized control scheme using localized position and RLNN controllers; (b) architecture of local reinforcement learning control system showing signals used for learning as dashed lines

Grahic Jump Location
+Reinforcement function for evaluating actuator performance
View Large  |  Save Figure  |  Download Slide (.ppt)
Reinforcement function for evaluating actuator performance
Figure 4

Reinforcement function for evaluating actuator performance

Grahic Jump Location
+Typical performance of noncommunicating hydraulic actuators on coordination task: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.
View Large  |  Save Figure  |  Download Slide (.ppt)
Typical performance of noncommunicating hydraulic actuators on coordination task: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.
Figure 5

Typical performance of noncommunicating hydraulic actuators on coordination task: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.

Grahic Jump Location
+Performance of noncommunicating hydraulic actuators on coordination task after learning: (a) rms interaction force; (b) actuator 1 trajectory correction ratio
View Large  |  Save Figure  |  Download Slide (.ppt)
Performance of noncommunicating hydraulic actuators on coordination task after learning: (a) rms interaction force; (b) actuator 1 trajectory correction ratio
Figure 6

Performance of noncommunicating hydraulic actuators on coordination task after learning: (a) rms interaction force; (b) actuator 1 trajectory correction ratio

Grahic Jump Location
+Typical performance on coordination task with communication: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.
View Large  |  Save Figure  |  Download Slide (.ppt)
Typical performance on coordination task with communication: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.
Figure 7

Typical performance on coordination task with communication: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.

Grahic Jump Location
+Actuator 1 trajectory correction ratios after learning using communication scheme
View Large  |  Save Figure  |  Download Slide (.ppt)
Actuator 1 trajectory correction ratios after learning using communication scheme
Figure 8

Actuator 1 trajectory correction ratios after learning using communication scheme

Grahic Jump Location
+Photograph of test rig upon which experiments were carried out. Inset: Close-up view of deformable object and load cell.
View Large  |  Save Figure  |  Download Slide (.ppt)
Photograph of test rig upon which experiments were carried out. Inset: Close-up view of deformable object and load cell.
Figure 9

Photograph of test rig upon which experiments were carried out. Inset: Close-up view of deformable object and load cell.

Grahic Jump Location
+Benchmark experimental performance on sinusoidal trajectory 22 before learning: (a) measured force; (b) position trajectory errors, x1d−x1 and x2d−x2
View Large  |  Save Figure  |  Download Slide (.ppt)
Benchmark experimental performance on sinusoidal trajectory 22 before learning: (a) measured force; (b) position trajectory errors, x1d−x1 and x2d−x2
Figure 10

Benchmark experimental performance on sinusoidal trajectory 22 before learning: (a) measured force; (b) position trajectory errors, x1d−x1 and x2d−x2

Grahic Jump Location
+Typical experimental performance of noncommunicating agents on coordination task: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.
View Large  |  Save Figure  |  Download Slide (.ppt)
Typical experimental performance of noncommunicating agents on coordination task: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.
Figure 11

Typical experimental performance of noncommunicating agents on coordination task: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.

Grahic Jump Location
+Typical experimental performance on coordination task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.
View Large  |  Save Figure  |  Download Slide (.ppt)
Typical experimental performance on coordination task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.
Figure 12

Typical experimental performance on coordination task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.

Grahic Jump Location
+Typical experimental performance with communication and increased object stiffness: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) learning; (2) after learning.
View Large  |  Save Figure  |  Download Slide (.ppt)
Typical experimental performance with communication and increased object stiffness: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) learning; (2) after learning.
Figure 13

Typical experimental performance with communication and increased object stiffness: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) learning; (2) after learning.

Grahic Jump Location
+Benchmark experimental performance on test trajectory with starting and stopping motions: (a) desired position; (b) measured force
View Large  |  Save Figure  |  Download Slide (.ppt)
Benchmark experimental performance on test trajectory with starting and stopping motions: (a) desired position; (b) measured force
Figure 14

Benchmark experimental performance on test trajectory with starting and stopping motions: (a) desired position; (b) measured force

Grahic Jump Location
+Typical experimental performance for trajectory following test task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) after pretraining on sinusoidal trajectory for 30s; (2) additional learning.
View Large  |  Save Figure  |  Download Slide (.ppt)
Typical experimental performance for trajectory following test task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) after pretraining on sinusoidal trajectory for 30s; (2) additional learning.
Figure 15

Typical experimental performance for trajectory following test task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) after pretraining on sinusoidal trajectory for 30s; (2) additional learning.

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
On-Line a Predictive Model of Cutting Force in Turning with 3 Axis Acceleration Transducer Using Neural Network
International Conference on Advanced Computer Theory and Engineering (ICACTE 2009)
Comparisons of Gaussian ARTMAP and Distributed Gaussian ARTMAP Classifiers: The Category Proliferation Problem
Intelligent Engineering Systems through Artificial Neural Networks, Volume 16
A Novel Approach for LFC and AVR of an Autonomous Power Generating System
International Conference on Mechanical Engineering and Technology (ICMET-London 2011)
A New Exploratory Neural Network Training Method
Intelligent Engineering Systems through Artificial Neural Networks, Volume 16
Fitting a Function and Its Derivative
Intelligent Engineering Systems Through Artificial Neural Networks, Volume 17
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In