Review Article

Tips on Stochastic Optimal Feedback Control and Bayesian Spatiotemporal Models: Applications to Robotics

[+] Author and Article Information
Jongeun Choi

Associate Professor
Department of Mechanical Engineering,
Department of Electrical and
Computer Engineering,
Michigan State University,
East Lansing, MI 48864
e-mail: jchoi@egr.msu.edu

Dejan Milutinović

Associate Professor
Computer Engineering Department,
University of California,
Santa Cruz,
Santa Cruz, CA 95064
e-mail: dejan@soe.ucsc.edu

1Corresponding author.

Contributed by the Dynamic Systems Division of ASME for publication in the JOURNAL OF DYNAMIC SYSTEMS, MEASUREMENT, AND CONTROL. Manuscript received February 14, 2014; final manuscript received September 22, 2014; published online October 21, 2014. Editor: J. Karl Hedrick.

J. Dyn. Sys., Meas., Control 137(3), 030801 (Oct 21, 2014) (10 pages) Paper No: DS-14-1068; doi: 10.1115/1.4028642 History: Received February 14, 2014; Revised September 22, 2014

This tutorial paper presents the expositions of stochastic optimal feedback control theory and Bayesian spatiotemporal models in the context of robotics applications. The presented material is self-contained so that readers can grasp the most important concepts and acquire knowledge needed to jump-start their research. To facilitate this, we provide a series of educational examples from robotics and mobile sensor networks.

Copyright © 2015 by ASME
Your Session has timed out. Please sign back in to continue.



Grahic Jump Location
Fig. 1

Fixed velocity two-wheel robot control problems: (a) Minimum expected time control (P1); (b) Distance keeping control (P2); (xx,xy)—the robot coordinates relative to the target which is at the origin, r—distance between the robot and the target, φ—robot heading angle, α—bearing angle and ν—velocity

Grahic Jump Location
Fig. 2

Solution of the minimum expected time problem (P1): (left panel) gray colored map of the value function V(r, α); black color at the absorbing boundary (A) indicates V(r, α) = 0 and the lighter shades depict longer expected times. The type of the boundary conditions is labeled by P—periodic, R—reflective, A—absorbing; (right panel) optimal feedback control; white u = 1 and gray u = –1.

Grahic Jump Location
Fig. 3

Solution of the distance keeping control problem (P2): (left panel) gray colored map of the value function V(r, α); darker shades correspond to smaller values of V(r, α) and lighter shades depict its larger values. The type of the boundary conditions are labeled by P—periodic, R—reflective; (right panel) optimal feedback control, white u = 1 and gray u = − 1.

Grahic Jump Location
Fig. 4

Each agent is driven by swarming and gradient ascent efforts based on its own recursively estimated field (with a model in Eq. (21)) via locally collected measurements by itself and its neighboring agents. The multiagent system locates peaks of an uncertain static field in a distributed and scalable manner [39].

Grahic Jump Location
Fig. 5

(a) Remotely controlled boat equipped with depth sensor and GPS and (b) experiment site with the sampling locations (shown as colored dots)

Grahic Jump Location
Fig. 6

Prediction with hyperparameters σf = 0.2, σ1 = 0.07, σ2 = 0. 07, and σw = 0. 03. (a) Estimated depth and (b) prediction error variance, with sampling positions shown as white crosses.

Grahic Jump Location
Fig. 7

Prediction with hyperparameters σf = 0.2, σ1 = 0.2, σ2 = 0. 2, and σw = 0. 03. (a) Estimated depth and (b) prediction error variance, with sampling positions shown as white crosses.

Grahic Jump Location
Fig. 8

The posterior distribution of β, σf2, σs, and σt at (a) t1 and (b) t20

Grahic Jump Location
Fig. 9

The prediction at (a) t1 and (b) t20 using the MCMC-based approach. The true fields are plotted in blue solid lines. The predicted fields are plotted in red dashed lines. The area between red dotted lines indicates the 95% confidence interval.




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In