Development of Markovian Models

The Hidden Markov Model (HMM) is a powerful statistical tool for modeling a sequence of data elements called the observation vectors. As such, extraction of patterns in time series data can be facilitated by a judicious selection and training of HMMs. In this section, a brief overview will be presented and the interested reader can find more details in numerous tutorials 

---

The second type of nonideal models takes into account the possible formation of donor-acceptor complexes between monomers. Essentially, along with individual entry of these latter into a polymer chain, the possibility arises for their addition to this chain as a binary complex. A theoretical analysis of copolymerization in the framework of this model revealed (Korolev and Kuchanov, 1982) that the statistics of the succession of units in macromolecules is not Markovian even at fixed monomer mixture composition in a reactor. Nevertheless, an approach based on the "labeling-erasing" procedure has been developed (Kuchanov et al., 1984), enabling the calculation of any statistical characteristics of such non-Markovian copolymers. 

The health consequences on the population must be estimated by taking into consideration the type of the accident and the population distribution in the area as a function of time. To calculate the variation of the spatial population density, we have developed a stochastic model that simulates the evacuation procedure. More precisely, we have adopted a Markovian type stochastic model to simulate the movement of the population (Georgiadou et al, 2006). 

In order to stndy the short time vibrational energy transfer behavior of a vibra-tionally excited system, we employ a non-Markovian time-dependent perturbation theory [83]. Onr approach builds on the successful application of Markovian time-dependent pertnrbation theory by Leitner and coworkers to explore heat flow in proteins and glasses, and Tokmakoff, Payer, and others, in modeling vibrational population relaxation of selected modes in larger molecules. In a separate chapter in this volnme, Leitner provides an overview of the development of normal mode-based methods, snch as the one employed here, for the study of energy flow in solids and larger molecnlar systems. 

We have developed a simple and physically clear picture of ARP in molecules in solution by careful examination of all the conditions needed for ARP. The relaxation effects were considered in the framework of the LZ model for random crossing of levels. The model enables us to include into consideration non-Markovian Gaussian-correlated noise. It explains all the numerical results obtained in Ref. [4]. 

Neither the maximum nor the descending branches of the upper curves, representing geminate recombination, are reproduced in the Markovian theory. It predicts the monotonous ion accumulation and still further decrease in the ionization quantum yield /. This is because the Markovian theory does not account for either static or subsequent nonstationary electron transfer. When ionization is under diffusional control, both these are faster than the final (Markovian) transfer. EM is a bit better in this respect. As a non-Markovian theory, it accounts at least for static ionization and qualitatively reproduces the maximum in the charge accumulation kinetics. However, the subsequent geminate recombination develops exponentially in EM because the kinematics of ion separation is oversimplified in this model. It roughly contradicts an actual diffusional separation of ions, characterized by numerous recontacts and the power dependence of long-time separation kinetics studied in a number of works [20,21,187],... 

Various methods have been developed that interpolate between the coherent and incoherent regimes (for reviews see, e.g. (3)-(5)). Well-known approaches use the stochastic Liouville equation, of which the Haken-Strobl-Reineker (3) model is an example, and the generalized master equation (4). A powerful technique, which in principle deals with all aspects of the problem, uses the reduced density matrix of the exciton subsystem, which is obtained by projecting out all degrees of freedom (the bath) from the total statistical operator (6). This reduced density operator obeys a closed non-Markovian (integrodifferential) equation with a memory kernel that includes the effects of (multiple) interactions between the excitons and the bath. In practice, one is often forced to truncate this kernel at the level of two interactions. In the Markov approximation, the resulting description is known as Redfield theory (7). 

A recent theoretical development is the use of NMR to study compositional heterogeneity. Two approaches can be used 1) perturbed Markovian (continuous) model (105), and 2) multi-conq)onent (discrete) model (99), Neiss and Cheng have applied the discrete model to the SEC-NMR data of alginates (69). In an earlier work, the NMR data of alginates have also been fitted to a continuous model (106). 

---