0
Research Papers

# Periodic-Node Graph-Based Framework for Stochastic Control of Small Aerial Vehicles

[+] Author and Article Information

Laboratory for Information
and Decision Systems,
Massachusetts Institute of Technology,
Cambridge, MA 02139
e-mail: aliagha@mit.edu

Saurav Agarwal

Department of Aerospace Engineering,
Texas A&M University,
College Station, TX 77840
e-mail: sauravag@tamu.edu

Suman Chakravorty

Associate Professor
Department of Aerospace Engineering,
Texas A&M University,
College Station, TX 77840
e-mail: schakrav@tamu.edu

Contributed by the Dynamic Systems Division of ASME for publication in the JOURNAL OF DYNAMIC SYSTEMS, MEASUREMENT, AND CONTROL. Manuscript received January 2, 2014; final manuscript received June 18, 2014; published online October 21, 2014. Assoc. Editor: Dejan Milutinovic.

J. Dyn. Sys., Meas., Control 137(3), 031002 (Oct 21, 2014) (14 pages) Paper No: DS-14-1004; doi: 10.1115/1.4027888 History: Received January 02, 2014; Revised June 18, 2014

## Abstract

This paper presents a strategy for stochastic control of small aerial vehicles under uncertainty using graph-based methods. In planning with graph-based methods, such as the probabilistic roadmap method (PRM) in state space or the information roadmaps (IRM) in information-state (belief) space, the local planners (along the edges) are responsible to drive the state/belief to the final node of the edge. However, for aerial vehicles with minimum velocity constraints, driving the system belief to a sampled belief is a challenge. In this paper, we propose a novel method based on periodic controllers, in which instead of stabilizing the belief to a predefined probability distribution, the belief is stabilized to an orbit (periodic path) of probability distributions. Choosing nodes along these orbits, the node reachability in belief space is achieved and we can form a graph in belief space that can handle higher order dynamics or nonstoppable systems (whose velocity cannot be zero), such as fixed-wing aircraft. The proposed method takes obstacles into account and provides a query-independent graph, since its edge costs are independent of each other. Thus, it satisfies the principle of optimality. Therefore, dynamic programming (DP) can be utilized to compute the best feedback on the graph. We demonstrate the method's performance on a unicycle robot and a six degrees of freedom (DoF) small aerial vehicle.

<>

## References

Astrom, K., 1965, “Optimal Control of Markov Decision Processes With Incomplete State Estimation,” J. Math. Anal. Appl., 10, pp. 174–205.
Kaelbling, L. P., Littman, M. L., and Cassandra, A. R., 1998, “Planning and Acting in Partially Observable Stochastic Domains,” Artif. Intell., 101, pp. 99–134.
Kavraki, L., Švestka, P., Latombe, J., and Overmars, M., 1996, “Probabilistic Roadmaps for Path Planning in High-Dimensional Configuration Spaces,” IEEE Trans. Rob. Autom., 12(4), pp. 566–580.
Amato, N., Bayazit, B., Dale, L., Jones, C., and Vallejo, D., 1998, “OBPRM: An Obstacle-Basaed PRM for 3D Workspaces,” International Workshop on the Algorithmic Foundations of Robotics, Houston, TX, March 4–6, pp. 155–168. Available at:
Karaman, S., and Frazzoli, E., 2011, “Sampling-Based Algorithms for Optimal Motion Planning,” Int. J. Rob. Res., 30(7), pp. 846–894.
Lavalle, S., and Kuffner, J., 2001, “Randomized Kinodynamic Planning,” Int. J. Rob. Res., 20, pp. 378–400.
Hsu, D., 2000, “Randomized Single-Query Motion Planning in Expansive Spaces,” Ph.D. thesis, Department of Computer Science, Stanford University, Stanford, CA.
Agha-mohammadi, A., Chakravorty, S., and Amato, N., 2011, “FIRM: Feedback Controller-Based Information-State RoadMap, A Framework for Motion Planning Under Uncertainty,” International Conference on Intelligent Robots and Systems, (IROS ), San Francisco, CA, Sept 25–30.
Agha-mohammadi, A., Chakravorty, S., and Amato, N., 2014, “FIRM: Sampling-Based Feedback Motion Planning Under Motion Uncertainty and Imperfect Measurements,” Int. J. Rob. Res., 33(2), pp. 268–304.
Prentice, S., and Roy, N., 2009, “The Belief Roadmap: Efficient Planning in Belief Space by Factoring the Covariance,” Int. J. Rob. Res., 28(11–12), pp. 1448–1465.
Huynh, V., and Roy, N., 2009, “icLQG: Combining Local and Global Optimization for Control in Information Space,” IEEE International Conference on Robotics and Automation, (ICRA), Kobe, Japan, May 12–17, pp. 2851-2858.
van den Berg, J., Abbeel, P., and Goldberg, K., 2011, “LQG-MP: Optimized Path Planning for Robots With Motion Uncertainty and Imperfect State Information,” Int. J. Rob. Res., 30(7), pp. 895–913.
Agha-mohammadi, A., Chakravorty, S., and Amato, N., 2012, “On the Probabilistic Completeness of the Sampling-Based Feedback Motion Planners in Belief Space,” IEEE International Conference on Robotics and Automation (ICRA ), Saint Paul, MN, May 14–18.
Agha-mohammadi, A., Agarwal, S., Mahadevan, A., Chakravorty, S., Tomkins, D., Denny, J., and Amato, N., 2014, “Robust Online Belief Space Planning in Changing Environments: Application to Physical Mobile Robots,” IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May 31–June 7.
Brockett, R. W., 1983, “Asymptotic Stability and Feedback Stabilization,” Differential Geometric Control Theory, R. W.Brockett, and H. J.Sussmann, eds., Birkhauser, Boston, pp. 181–191.
Bertsekas, D., 2007, Dynamic Programming and Optimal Control, 3rd ed., Athena Scientific, Cambridge, MA.
Thrun, S., Burgard, W., and Fox, D., 2005, Probabilistic Robotics, MIT, Cambridge, MA.
Sutton, R., Precup, D., and Singh, S., 1999, “Between MDPS and Semi-MDPS: A Framework for Temporal Abstraction in Reinforcement Learning,” Artif. Intell., 112, pp. 181–211.
Kumar, P. R., and Varaiya, P. P., 1986, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice-Hall, Englewood Cliffs, NJ.
Bittanti, S., and Colaneri, P., 2009, Periodic Systems Filtering and Control, Springer-Verlag, New York. Available at: .
Colaneri, P., Scattolini, R., and Schiavoni, N., 1992, “LQG Optimal Control for Multirate Samled-Data Systems,” IEEE Trans. Autom. Control, 37(5), pp. 675–682.
Bittanti, S., Bolzern, P., De Nicolao, G., and Piroddi, L., 1994, “Representation, Prediction and Identification of Cyclostationary Processes,” Cyclostationarity in Communications and Signal Processing, W. A.Gardner, ed., IEEE, New York.
Doucet, A., de Freitas, J., and Gordon, N., 2001, Sequential Monte Carlo Methods in Practice, Springer, New York.
Simon, D., 2006, Optimal State Estimation: Kalman, H-Infinity, and Nonlinear Approaches, John Wiley and Sons, Inc., Hoboken, NJ. Available at:
Crassidis, J., and Junkins, J., 2004, Optimal Estimation of Dynamic Systems, Chapman & Hall/CRC, Boca Raton, FL.
Anderson, R. P., Bakolas, E., Milutinović, D., and Tsiotras, P., 2013, “Optimal Feedback Guidance of a Small Aerial Vehicle in a Stochastic Wind,” J. Guid. Control Dyn., 36, pp. 975–985.
Agha-mohammadi, A., Agarwal, S., and Chakravorty, S., 2014, “Periodic-Node Graph-Based Framework for Stochastic Control of Small Aerial Vehicles,” Technical Report, Department of Aerospace Engineering, Texas A&M University. Report No. TAMU-EDPL-TR-2014-001.
Meyn, S., and Tweedie, R. L., 2009, Markov Chains and Stochastic Stability, 2nd ed., Cambridge University, Cambridge, UK.

## Figures

Fig. 1

A simple PNPRM with three orbits, twelve nodes, and two edges

Fig. 2

bαl≡(vαl,Pkαl) is the center of belief nodes corresponding to the nodes shown in Fig. 1, where Pkαl's are shown by their 3σ-ellipse. As an example of a FIRM node, the magnified version of B2j, which is a small neighborhood centered at b2j, is shown in the dotted box, where the shaded region (blue) around the outer ellipse boundary depicts the covariance neighborhood and the shaded circular region (green) around the center point denotes the mean neighborhood.

Fig. 3

(a) Five orbits (T = 100) and corresponding periodic estimation covariances as the SPPS solution of DPRE in Eq. (14). (b) Sample covariance convergence on an orbit (T = 20) under PLQG. Red ellipses are the solution of DPRE and green ellipses are the evolution of estimation covariance. The initial covariance is three times bigger than the SPPS solution of DPRE, i.e., P0 = 3P0.

Fig. 4

A sample PNPRM with circular orbits. Number of each orbit is written at its center. Nine landmarks (black stars) and obstacles (gray polygons) are also shown. The directions of motion on orbits and edges are shown by little triangles with a cross in their heading direction. (a) Orbits 2 and 7 (distinguished in black) are the start and goal orbits, respectively. Shortest path (in green, flowing through Orbits 2,3,4,5,6,7) and the most-likely path (in red, flowing through orbit 2,1,9,10,8,20,19,14,15,16,7) under the FIRM policy are also shown. (b) Assuming on each orbit, there exists a single node, the feedback πg is visualized for all FIRM nodes.

Fig. 5

The aircraft attached body-fixed frame and ground frame

Fig. 6

The PNPRM in 3D showing the orbits and edges

Fig. 8

Feedback πg is shown with orbit 3 as the goal orbit. Starting from orbit 2, the most-likely path (in red, flowing through orbits 2, 3) is shown (a) from the top view and (b) in the 3D environment.

Fig. 9

Feedback πg is shown with orbit 7 as the goal orbit. Starting from orbit 3, the most-likely path (in red, flowing through orbits 12,9,8) is shown (a) from the top view and (b) in the 3D environment.

Fig. 7

Feedback πg is shown with orbit 2 as the goal orbit. (a) Starting from orbit 1, the shortest path (in green, flowing through orbits 1,4,5,3,2) and the most-likely path (in red, flowing through orbits 1,7,8,10,16,15,2) are shown from the top view (b) The shortest path (green) and the most-likely path (red) are shown in the 3D environment.

## Discussions

Some tools below are only available to our subscribers or users with an online account.

### Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related Proceedings Articles
Related eBook Content
Topic Collections