0

### Editorial

J. Dyn. Sys., Meas., Control. 2014;137(3):030201-030201-1. doi:10.1115/1.4028707.
FREE TO VIEW

The focus of the special issue is on robotic systems in which the influence of stochastic processes cannot be ignored and needs to be anticipated, for example, the presence of stochastic drifts, propagated (estimation) uncertainty in dynamics, or stochastic components in robotic system algorithms. In all these cases, stochastic processes are essential as a natural model for the uncertainty, a family of possible dynamic system evolutions, or algorithm executions.

Commentary by Dr. Valentin Fuster

### Review Article

J. Dyn. Sys., Meas., Control. 2014;137(3):030801-030801-10. doi:10.1115/1.4028642.

This tutorial paper presents the expositions of stochastic optimal feedback control theory and Bayesian spatiotemporal models in the context of robotics applications. The presented material is self-contained so that readers can grasp the most important concepts and acquire knowledge needed to jump-start their research. To facilitate this, we provide a series of educational examples from robotics and mobile sensor networks.

Commentary by Dr. Valentin Fuster

### Research Papers

J. Dyn. Sys., Meas., Control. 2014;137(3):031001-031001-8. doi:10.1115/1.4028552.

We propose an approach to the problem of computing a minimum-time tour through a series of waypoints for a Dubins vehicle in the presence of stochasticity. In this paper, we explicitly account for kinematic nonlinearities, the stochastic drift of the vehicle, the stochastic motion of the targets, and the possibility for the vehicle to service each of the targets or waypoints, leading to a new version of the Dubins vehicle traveling salesperson problem (TSP). Based on the Hamilton–Jacobi–Bellman (HJB) equation, we first compute the minimum expected time feedback control to reach one waypoint. Next, minimum expected times associated with the feedback control are used to construct and solve a TSP. We provide numerical results illustrating our solution, analyze how the stochasticity affects the solution, and consider the possibility for on-line recomputation of the waypoint ordering in a receding-horizon manner.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031002-031002-14. doi:10.1115/1.4027888.

This paper presents a strategy for stochastic control of small aerial vehicles under uncertainty using graph-based methods. In planning with graph-based methods, such as the probabilistic roadmap method (PRM) in state space or the information roadmaps (IRM) in information-state (belief) space, the local planners (along the edges) are responsible to drive the state/belief to the final node of the edge. However, for aerial vehicles with minimum velocity constraints, driving the system belief to a sampled belief is a challenge. In this paper, we propose a novel method based on periodic controllers, in which instead of stabilizing the belief to a predefined probability distribution, the belief is stabilized to an orbit (periodic path) of probability distributions. Choosing nodes along these orbits, the node reachability in belief space is achieved and we can form a graph in belief space that can handle higher order dynamics or nonstoppable systems (whose velocity cannot be zero), such as fixed-wing aircraft. The proposed method takes obstacles into account and provides a query-independent graph, since its edge costs are independent of each other. Thus, it satisfies the principle of optimality. Therefore, dynamic programming (DP) can be utilized to compute the best feedback on the graph. We demonstrate the method's performance on a unicycle robot and a six degrees of freedom (DoF) small aerial vehicle.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031003-031003-10. doi:10.1115/1.4027979.

This work develops and analyzes a control algorithm for an unmanned aerial vehicle (UAV) to circumnavigate an unknown target at a fixed radius when the UAV is unable to determine its location and heading. Using a relationship between range-rate and bearing angle (from the target), we formulate a control algorithm that uses the range-rate as a proxy for the bearing angle and adjusts the heading of the UAV accordingly. We consider the addition of measurement errors and model the system with a stochastic differential equation to carry out the analysis. A recurrence result is proven, establishing that the UAV will reach a neighborhood of the desired orbit in finite time, and a mollified control is presented to eliminate a portion of the recurrent set about the origin. Simulation studies are presented to support the analysis and compare the performance against other algorithms for the circumnavigation task.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031004-031004-9. doi:10.1115/1.4027892.

Autonomous robot networks are an effective tool for monitoring large-scale environmental fields. This paper proposes distributed control strategies for localizing the source of a noisy signal, which could represent a physical quantity of interest such as magnetic force, heat, radio signal, or chemical concentration. We develop algorithms specific to two scenarios: one in which the sensors have a precise model of the signal formation process and one in which a signal model is not available. In the model-free scenario, a team of sensors is used to follow a stochastic gradient of the signal field. Our approach is distributed, robust to deformations in the group geometry, does not necessitate global localization, and is guaranteed to lead the sensors to a neighborhood of a local maximum of the field. In the model-based scenario, the sensors follow a stochastic gradient of the mutual information (MI) between their expected measurements and the expected source location in a distributed manner. The performance is demonstrated in simulation using a robot sensor network to localize the source of a wireless radio signal.

Topics: Sensors , Algorithms , Signals
Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031005-031005-11. doi:10.1115/1.4028117.

In this paper, we consider a stochastic deployment problem, where a robotic swarm is tasked with the objective of positioning at least one robot at each of a set of pre-assigned targets while meeting a temporal deadline. Travel times and failure rates are stochastic but related, inasmuch as failure rates increase with speed. To maximize chances of success while meeting the deadline, a control strategy has therefore to balance safety and performance. Our approach is to cast the problem within the theory of constrained Markov decision processes (CMDPs), whereby we seek to compute policies that maximize the probability of successful deployment while ensuring that the expected duration of the task is bounded by a given deadline. To account for uncertainties in the problem parameters, we consider a robust formulation and we propose efficient solution algorithms, which are of independent interest. Numerical experiments confirming our theoretical results are presented and discussed.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031006-031006-6. doi:10.1115/1.4027828.

A control strategy is employed that modifies the stochastic escape times from one basin of attraction to another in a model of a double-gyre flow. The system studied captures the behavior of a large class of fluid flows that circulate and have multiple almost invariant sets. In the presence of noise, a particle in one gyre may randomly switch to an adjacent gyre due to a rare large fluctuation. We show that large fluctuation theory may be applied for controlling autonomous agents in a stochastic environment, in fact leveraging the stochasticity to the advantage of switching between regions of interest and concluding that patterns may be broken or held over time as the result of noise. We demonstrate that a controller can effectively manipulate the probability of a large fluctuation; this demonstrates the potential of optimal control strategies that work in combination with the endemic stochastic environment. To demonstrate this, stochastic simulations and numerical continuation are employed to tie together experimental findings with predictions.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031007-031007-11. doi:10.1115/1.4028148.

In this paper, we propose distributed Gaussian process regression (GPR) for resource-constrained distributed sensor networks under localization uncertainty. The proposed distributed algorithm, which combines Jacobi over-relaxation (JOR) and discrete-time average consensus (DAC), can effectively handle localization uncertainty as well as limited communication and computation capabilities of distributed sensor networks. We also extend the proposed method hierarchically using sparse GPR to improve its scalability. The performance of the proposed method is verified in numerical simulations against the centralized maximum a posteriori (MAP) solution and a quick-and-dirty solution. We show that the proposed method outperforms the quick-and-dirty solution and achieve an accuracy comparable to the centralized solution.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031008-031008-14. doi:10.1115/1.4028589.

We develop and implement a framework to address autonomous surveillance problems with a collection of pan-tilt (PT) cameras. Using tools from stochastic reachability with random sets, we formulate the problems of target acquisition, target tracking, and acquisition while tracking as reach-avoid dynamic programs for Markov decision processes (MDPs). It is well known that solution methods for MDP problems based on dynamic programming (DP), implemented by state space gridding, suffer from the curse of dimensionality. This becomes a major limitation when one considers a network of PT cameras. To deal with larger problems we propose a hierarchical task allocation mechanism that allows cameras to calculate reach-avoid objectives independently while achieving tasks collectively. We evaluate the proposed algorithms experimentally on a setup involving industrial PT cameras and mobile robots as targets.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031009-031009-11. doi:10.1115/1.4028035.

This paper presents a hierarchical approach for estimating the mission feasibility, i.e., the probability of mission completion (PoMC), for mobile robotic systems operating in stochastic environments. Mobile robotic systems rely on onboard energy sources that are expended due to stochastic interactions with the environment. Resultantly, a bivariate distribution comprised of energy source (e.g., battery) run-time and mission time marginal distributions can be shown to represent a mission process that characterizes the distribution of all possible missions. Existing methodologies make independent stochastic predictions for battery run-time and mission time. The approach presented makes use of the marginal predictions, as prediction pairs, to allow for Bayesian correlation estimation and improved process characterization. To demonstrate both prediction accuracy and mission classification gains, the proposed methodology is validated using a novel experimental testbed that enables repeated battery discharge studies to be conducted as a small robotic ground vehicle traverses stochastic laboratory terrains.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031010-031010-8. doi:10.1115/1.4028553.

Distributed optimization methods have been used extensively in multirobot task allocation (MRTA) problems. In distributed optimization domain, most of the algorithms are developed for solving convex optimization problems. However, for complex MRTA problems, the cost function can be nonconvex and multimodal in nature with more than one minimum or maximum points. In this paper, an effort has been made to address these complex MRTA problems with multimodal cost functions in a distributed manner. The approach used in this paper is a distributed primal–dual interior point method where noise is added in the search direction as a mechanism to allow the algorithm to escape from suboptimal solutions. The search direction from the distributed primal–dual interior point method and the weighted variable updates help in the generation of feasible primal and dual solutions and in faster convergence while the noise added in the search direction helps in avoiding local optima. The optimality and the computation time of this proposed method are compared with that of the genetic algorithm (GA) and the numerical results are provided in this paper.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031011-031011-8. doi:10.1115/1.4027853.

Vision guided robots have become an important element in the manufacturing industry. In most current industrial applications, vision guided robots are controlled by a look-then-move method. This method cannot support many new emerging demands which require real-time vision guidance. Challenge comes from the speed of visual feedback. Due to cost limit, industrial robot vision systems are subject to considerable latency and limited sampling rate. This paper proposes new algorithms to address this challenge by compensating the latency and slow sampling of visual feedback so that real-time vision guided robot control can be realized with satisfactory performance. Statistical learning methods are developed to model the pattern of target's motion adaptively. The learned model is used to recover visual measurement from latency and slow sampling. The imaging geometry of the camera and all-dimensional motion of the target are fully considered. Tests are conducted to provide evaluation from different aspects.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):031012-031012-10. doi:10.1115/1.4027871.

In this paper, we formulate the manipulator Jacobian matrix in a probabilistic framework based on the random matrix theory (RMT). Due to the limited available information on the system fluctuations, the parametric approaches often prove to be inadequate to appropriately characterize the uncertainty. To overcome this difficulty, we develop two RMT-based probabilistic models for the Jacobian matrix to provide systematic frameworks that facilitate the uncertainty quantification in a variety of complex robotic systems. One of the models is built upon direct implementation of the maximum entropy principle that results in a Wishart random perturbation matrix. In the other probabilistic model, the Jacobian matrix is assumed to have a matrix-variate Gaussian distribution with known mean. The covariance matrix of the Gaussian distribution is obtained at every time point by maximizing a Shannon entropy measure (subject to Jacobian norm and covariance positive semidefiniteness constraints). In contrast to random variable/vector based schemes, the benefits of the proposed approach now include: (i) incorporating the kinematic configuration and complexity in the probabilistic formulation; (ii) achieving the uncertainty model using limited available information; (iii) taking into account the working configuration of the robotic systems in characterization of the uncertainty; and (iv) realizing a faster simulation process. A case study of a 2R serial manipulator is presented to highlight the critical aspects of the process.

Commentary by Dr. Valentin Fuster

### Technical Brief

J. Dyn. Sys., Meas., Control. 2014;137(3):034501-034501-7. doi:10.1115/1.4027876.

This paper addresses the problem of goal-directed robot path planning in the presence of uncertainties that are induced by bounded environmental disturbances and actuation errors. The offline infinite-horizon optimal plan is locally updated by online finite-horizon adaptive replanning upon observation of unexpected events (e.g., detection of unanticipated obstacles). The underlying theory is developed as an extension of a grid-based path planning algorithm, called $ν⋆$, which was formulated in the framework of probabilistic finite state automata (PFSA) and language measure from a control-theoretic perspective. The proposed concept has been validated on a simulation test bed that is constructed upon a model of typical autonomous underwater vehicles (AUVs) in the presence of uncertainties.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):034502-034502-7. doi:10.1115/1.4027950.

This paper investigates global uncertainty propagation and stochastic motion planning for the attitude kinematics of a rigid body. The Fokker–Planck equation on the special orthogonal group is numerically solved via noncommutative harmonic analysis to propagate a probability density function along flows of the attitude kinematics. Based on this, a stochastic optimal control problem is formulated to rotate a rigid body while avoiding obstacles within uncertain environments in an optimal fashion. The proposed intrinsic, geometric formulation does not require the common assumption that uncertainties are Gaussian or localized. It can be also applied to complex rotational maneuvers of a rigid body without singularities in a unified way. The desirable properties are illustrated by numerical examples.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):034503-034503-8. doi:10.1115/1.4028084.

The dynamics of guided projectile systems are inherently stochastic in nature. While deterministic control algorithms such as impact point prediction (IPP) may prove effective in many scenarios, the probability of impacting obstacles and constrained areas within an impact zone cannot be accounted for without accurate uncertainty modeling. A stochastic model predictive guidance algorithm is developed, which incorporates nonlinear uncertainty propagation to predict the impact probability density in real-time. Once the impact distribution is characterized, the guidance system aim point is computed as the solution to an optimization problem. The result is a guidance law that can achieve minimum miss distance while avoiding impact area constraints. Furthermore, the acceptable risk of obstacle impact can be quantified and tuned online. Example trajectories and Monte Carlo simulations demonstrate the effectiveness of the proposed stochastic control formulation in comparison to deterministic guidance schemes.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):034504-034504-9. doi:10.1115/1.4028353.

This technical brief summarizes and extends our recently introduced control framework for stochastically allocating a swarm of robots among boundaries of circular regions. As in the previous work, a macroscopic model of the swarm population dynamics is used to synthesize robot control policies that establish and maintain stable predictable team sizes around region boundaries. However, this extension shows that the control strategy can be implemented with no robot-to-robot communication. Moreover, target team sizes can vary across different types of regions, where a region's type is a subjective characteristic that only needs to be detectable by each individual robot. Thus, regions of one type may have a higher equilibrium team size than regions of another type. In other work that predicts and controls stochastic swarm behaviors using macroscopic models, the equilibrium allocations of the swarm are sensitive to changes in the mean robot encounter rates with objects in the environment. Thus, in those works, as the swarm density or number of objects changes, the control policies on each robot must be retuned to achieve the desired allocations. However, our approach is insensitive to changes in encounter rate and therefore requires no retuning as the environment changes. In this extension, we validate these claims and show how the convergence rate to the target equilibrium allocations can be controlled in swarms with a sufficiently large free-robot population. Furthermore, we demonstrate how our framework can be used to experimentally measure the rates of robot encounters with occupied and unoccupied sections of region boundaries. Thus, our method can be viewed both as an encounter-rate-independent allocation strategy as well as a tool for accurately measuring encounter rates when using other swarm control strategies that depend on them.

Commentary by Dr. Valentin Fuster
J. Dyn. Sys., Meas., Control. 2014;137(3):034505-034505-6. doi:10.1115/1.4028162.

Many robotic applications need an accurate, robust, and fast estimation of finger pose. We present a novel finger pose estimation method using a motion capture system. The method combines a system identification stage and a state tracking stage in a unified framework. The system identification stage develops an accurate model of a finger, and the state tracking stage tracks the finger pose with the extended Kalman filter (EKF) algorithm based on the model obtained in the system identification stage. The algorithm is validated by simulation, and experiments with a human subject and a robotic finger. The experimental results show that the method can robustly estimate the finger pose at a high frequency (greater than 1 kHz) in the presence of measurement noise, occlusion of markers, and fast movement.

Commentary by Dr. Valentin Fuster