* Stochastic process
– Def 1. Mathematically, a stochastic process is usually defined as an indexed collection of random variables.
– Def 2. A probabilistic model of a system that evolves randomly. If the system is observed at time point n=0,1,2,…, and Xn is the state of the system at time n, then {Xn, n>=0} is a stochastic process describing it.

* Hidden Markov Model
– Observation is a probabilistic function of the state
– i.e., the resulting model (which is called a hidden Markov Model) is a doubly embedded stochastic process with an underlying stochastic process that is not observable (it is hidden)
– But can only be observed through another set of stochastic processes that produce the sequence of observations.

* References
– Lawrence R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” In Proceedings of the IEEE, Vol. 77, No. 2, February 1989
– Vidyadhar G. Kulkarni, Modeling and Analysis of Stochastic Systems, CHAPMAN&HALL, 1995
– http://en.wikipedia.org/wiki/Stochastic_process#Definition