0
TECHNICAL PAPERS

Decentralized Coordinated Motion Control of Two Hydraulic Actuators Handling a Common Object

[+] Author and Article Information
Mark Karpenko

Department of Mechanical and Manufacturing Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6, Canada

Nariman Sepehri1

Department of Mechanical and Manufacturing Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6, Canadanariman@cc.umanitoba.ca

John Anderson

Department of Computer Science, University of Manitoba, Winnipeg, Manitoba, R3T 5V6, Canada

1

Corresponding author.

J. Dyn. Sys., Meas., Control 129(5), 729-741 (Jan 25, 2007) (13 pages) doi:10.1115/1.2764516 History: Received June 23, 2006; Revised January 25, 2007

In this paper, reinforcement learning is applied to coordinate, in a decentralized fashion, the motions of a pair of hydraulic actuators whose task is to firmly hold and move an object along a specified trajectory under conventional position control. The learning goal is to reduce the interaction forces acting on the object that arise due to inevitable positioning errors resulting from the imperfect closed-loop actuator dynamics. Each actuator is therefore outfitted with a reinforcement learning neural network that modifies a centrally planned formation constrained position trajectory in response to the locally measured interaction force. It is shown that the actuators, which form a multiagent learning system, can learn decentralized control strategies that reduce the object interaction forces and thus greatly improve their coordination on the manipulation task. However, the problem of credit assignment, a common difficulty in multiagent learning systems, prevents the actuators from learning control strategies where each actuator contributes equally to reducing the interaction force. This problem is resolved in this paper via the periodic communication of limited local state information between the reinforcement learning actuators. Using both simulations and experiments, this paper examines some of the issues pertaining to learning in dynamic multiagent environments and establishes reinforcement learning as a potential technique for coordinating several nonlinear hydraulic manipulators performing a common task.

Copyright © 2007 by American Society of Mechanical Engineers
Your Session has timed out. Please sign back in to continue.

References

Figures

Grahic Jump Location
Figure 1

Schematic of a typical hydraulic actuator for mathematical modeling

Grahic Jump Location
Figure 2

Schematic of coordinated positioning task: (a) definition of reference trajectories x1d and x2d with respect to desired object position xd; (b) deformation of object due to relative actuator positioning error

Grahic Jump Location
Figure 3

Schematic of coordinated positioning system: (a) decentralized control scheme using localized position and RLNN controllers; (b) architecture of local reinforcement learning control system showing signals used for learning as dashed lines

Grahic Jump Location
Figure 4

Reinforcement function for evaluating actuator performance

Grahic Jump Location
Figure 5

Typical performance of noncommunicating hydraulic actuators on coordination task: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.

Grahic Jump Location
Figure 6

Performance of noncommunicating hydraulic actuators on coordination task after learning: (a) rms interaction force; (b) actuator 1 trajectory correction ratio

Grahic Jump Location
Figure 7

Typical performance on coordination task with communication: (a) interaction force; (b) trajectory corrections, Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) before learning; (2) learning; (3) after learning.

Grahic Jump Location
Figure 8

Actuator 1 trajectory correction ratios after learning using communication scheme

Grahic Jump Location
Figure 9

Photograph of test rig upon which experiments were carried out. Inset: Close-up view of deformable object and load cell.

Grahic Jump Location
Figure 10

Benchmark experimental performance on sinusoidal trajectory 22 before learning: (a) measured force; (b) position trajectory errors, x1d−x1 and x2d−x2

Grahic Jump Location
Figure 11

Typical experimental performance of noncommunicating agents on coordination task: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.

Grahic Jump Location
Figure 12

Typical experimental performance on coordination task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2; (c) position trajectory errors, x1d−x1 and x2d−x2. Legend: (1) learning; (2) after learning.

Grahic Jump Location
Figure 13

Typical experimental performance with communication and increased object stiffness: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) learning; (2) after learning.

Grahic Jump Location
Figure 14

Benchmark experimental performance on test trajectory with starting and stopping motions: (a) desired position; (b) measured force

Grahic Jump Location
Figure 15

Typical experimental performance for trajectory following test task with communication: (a) measured force; (b) trajectory corrections Δx1 and Δx2. Legend: (1) after pretraining on sinusoidal trajectory for 30s; (2) additional learning.

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In