Research Papers

Estimation of State Transition Probabilities in Asynchronous Vector Markov Processes

[+] Author and Article Information
Waleed A. Farahat

Department of Mechanical Engineering,  Massachusetts Institute of Technology, Cambridge, MA 02139wfarahat@mit.edu

H. Harry Asada

Department of Mechanical Engineering,  Massachusetts Institute of Technology, Cambridge, MA 02139asada@mit.edu

J. Dyn. Sys., Meas., Control 134(6), 061003 (Sep 13, 2012) (14 pages) doi:10.1115/1.4006087 History: Received February 25, 2010; Accepted November 27, 2011; Published September 13, 2012

Vector Markov processes (also known as population Markov processes) are an important class of stochastic processes that have been used to model a wide range of technological, biological, and socioeconomic systems. The dynamics of vector Markov processes are fully characterized, in a stochastic sense, by the state transition probability matrix P . In most applications, P has to be estimated based on either incomplete or aggregated process observations. Here, in contrast to established methods for estimation given aggregate data, we develop Bayesian formulations for estimating P from asynchronous aggregate (longitudinal) observations of the population dynamics. Such observations are common, for example, in the study of aggregate biological cell population dynamics via flow cytometry. We derive the Bayesian formulation, and show that computing estimates via exact marginalization are, generally, computationally expensive. Consequently, we rely on Monte Carlo Markov chain sampling approaches to estimate the posterior distributions efficiently. By explicitly integrating problem constraints in these sampling schemes, significant efficiencies are attained. We illustrate the algorithm via simulation examples and show that the Bayesian estimation schemes can attain significant advantages over point estimates schemes such as maximum likelihood.

Copyright © 2012 by American Society of Mechanical Engineers
Your Session has timed out. Please sign back in to continue.



Grahic Jump Location
Figure 1

A vector Markov process consists of multiple agents or cells, each exhibiting Markovian dynamics

Grahic Jump Location
Figure 2

Illustration of panel data S and longitudinal data Θ for a two-state system for both synchronous and asynchronous transitions. (a) and (b): Cell color (gray or white) represents the state of each cell (state 1 or state 2) as it evolves over time. (c) and (d): Bar heights represent the total number of cells (aggregate data) in each state for each time step. The vertical lines represent observation time points.

Grahic Jump Location
Figure 3

A Poisson arrival process can be used to model asynchronous state transitions in the population. The plots illustrate the cellular transitions (for a two-state case), the Poisson distributions characterizing the number of transitions in Δt = 1, and the exponential distributions for the interarrival times. The plots are shown for values of λ = 1/3 (first row), λ = 1 (second row), and λ = 3 (third row).

Grahic Jump Location
Figure 4

Example for constrained low dimensional Gaussian sampling. The high dimensional space (f -space) is two dimensional. The two-dimensional Gaussian distribution is illustrated by the unimodal surface.The constraint is shown by the vertical upright plane, and cuts through the two-dimensional Gaussian, revealing a lower dimensional Gaussian (1D). This lower dimensional Gaussian is sampled as shown by the black asterisks.

Grahic Jump Location
Figure 5

Illustration of Gibbs sampling scheme to efficiently sample from Gaussian distributions subject to inequality constraints. (a) Gaussian distribution contours shown in problem space. The feasible region satisfying all inequality constraints is shaded in gray. Samples are drawn from the distribution irrespective of the inequality constraints (asterisks). Only the samples satisfying the inequality constraints are picked (indicated by circles in addition to asterisks) and retained. (b) Distribution, data, and constraints of subfigure (a) but are shown under the transformation. (c) In the transformed space, samples are drawn from the constrained distribution via Gibbs sampling. All samples satisfy the inequality constraints. (d) Samples are transformed back to problem space, maintaining feasibility.

Grahic Jump Location
Figure 6

Estimation from panel data based on uniform (noninformative) priors. Posterior shown on right. Asterisks represent maximum likelihood estimates, whereas the white lines represent the true transition probabilities used to generate the data.

Grahic Jump Location
Figure 7

Progression of the estimation algorithm in time as the estimates converge to the true value, represented by the point (1/3, 1/5). The figure illustrates convergence when priors are generic and mutlimodal in character.

Grahic Jump Location
Figure 8

Estimation from asynchronous data. Uniform (noninformative) priors shown on top. Posterior shown on bottom. The asterisks represent maximum likelihood estimates, whereas the lines intersecting at (0.33, 0.2) represent the true transition probabilities used to generate the data. The lines intersecting at (0.65, 0.58) represent the transformation of the true transition probabilities to P Δt .

Grahic Jump Location
Figure 9

Effects of Monte Carlo sample size on solution parameters for an example four dimensional problem. (a) The variance of error with sample size is estimated from ten repetitions of the solution runs. With increasing number of Gibbs samples, variance approaches 1/N rate of decrease indicated by the dotted line. (b) Computation time, as expected, increases linearly with number of Gibbs samples acquired.

Grahic Jump Location
Figure 10

Comparison of error estimates: Bayesian versus maximum likelihood estimates. Simulation parameters are: N = 100 and T = 5. In this particular instance, g(θ (t)) =θ 1 (t)3 · θ2 (t).



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In