Semi-Markov Processes

In the mathematical literature, X (t ) is called a semi-Markov process associated with the two-component Markov chain X , T ), a Markov renewal process [218]. As discussed in Sect. 2.3, the CTRW model is a standard approach for studying anomalous diffusion [298]. 

As mentioned on page 61, CTRWs are known as semi-Markov processes in the mathematical literature. In this section we provide a brief account of semi-Markov processes. They were introduced by P. Levy and W. L. Smith [253,415]. Recall that for a continuous-time Markov chain, the transitions between states at random times T are determined by the discrete chain X with the transition matrix H = (hij). The waiting time = T - for a given state i is exponentially distributed with the transition rate k , which depends only on the current state i. The natural generalization is to allow arbitrary distributions for the waiting times. This leads to a semi-Markov process. The reason for such a name is that the underlying process is a two-component Markov chain (X , T ). Here the random sequence X represents the state at the th transition, and T is the time of the nth transition. Obviously,... 

The standard continuous-time Markov chain is a special case of a semi-Markov process with the transition kernel... 

Janssen, J., Manca, R. Applied Semi-Markov Processes. Springer, New York (2006)... 

A semi-Markov process (SMP) can be understood as a probabilistic model for which the future behavior is conditional on the sojourn times (x) that are, in turn random variables dependent on the current state and on the state to which SMP will transit next. According to Ouhbi Limnios (2003), SMP are more flexible models than ordinary Markov processes as it is no longer required to assume that sojourn times are exponentially distributed. 

Moura, M. C., Firmino, E R. A., Droguett, E. L. Jacinto, C. M. 2008. Optical Monitoring System Availability Optimization via Semi-Markov Processes and Genetic Algorithms. In Proceedings of the 54th Annual Reliability Maintainability Symposium (RAMS), Las Vegas, USA, 28-31 January 2008. 

Ouhbi, B. Limnios, N. 2003. Nonparametric reliability estimation of semi-Markov processes. Journal of Statistical Planning and Inference, 109(1) 155-165. 

In extreme cases, such as when the number of source processes is small or when many failures are allowed to occur before a repair is made, the superposed process will have to be modelled by a semi-Markov process based on the rate-optimal approximation of Torab and Kamen (2001). 

The model of the operation process of the complex technical system with the distinguished their operation states is proposed in (Kolowrocki Soszynska, 2008). The semi-markov process is used to construct a general probabilistic model of the considered complex industrial system operation process. To construct this model there were defined the vector of the probabilities of the system initial operation states, the matrix of the probabilities of transitions between the operation states, the matrix of the distribution functions and the matrix of the density functions of the conditional sojourn times in the particular operation states. To describe the system operation process conditional sojourn times in the particular operation states the uniform distribution, the triangular distribution, the double trapezium distribution, the quasi- trapezium distribution, the exponential distribution, the WeibuU s... 

A faster numerical procedure for solving non-homogeneous semi-Markov processes... 

Continuous time non-homogeneous semi-Markov processes (CTNHSMP) are powerful modeling tools, especially in the reliability field (as exemplified in Janssen Manca (2007)). According to Becker et al. (2000), CTNHSMP are considered as approaches to model reliability characteristics of components or small systems with complex test and maintenance strategies. 

The semi-Markov process is therefore considered as non-homogeneous so that this deterioration process may be adequately addressed. Therefore, the required data to estimate the system availability via this CTNHSMP model are the parameters pyi ) and... 

To conclude, two important limitations of this work deserve attention. Firstly discussing semi-Markov processes in general, we have the well-known and already cited difficulty in obtaining the requisite data to analyze semi-Markov processes on the non-homogeneous environment. On that, El-Gohary (2004) presents max-iminn likelihood and Bayes estimates of the parameters included in a semi-Markov reliability model of three states. 

Janssen, J. Manca, R. 2001. Numerical solution of Non Homogeneous semi Markov processes in Transient Case. Methodology and Computing in Applied Probability, 3(271-293). 

Moura, M. C. Droguett, E. L. 2009. Mathematical formulation and numerical treatment based on transition frequency densities and quadrature methods for non-homogeneous semi-Markov processes. Reliability Engineering System Safety, 94(2) 342-349. 

Pievatolo A., Tironi E. Valade I. 2004. Semi-Markov Processes for Power System ReliabilityAssessment with Application to Uninterruptible Power Supply, IEEE Uans-actions on Power Apparatus and Systems, Vol. 19, No. 5, pp. 1326-1333. 

The analysis and characterization of the resilience of critical infrastructures is receiving a lot of attention from the scientific and technical community. For example, (Nozick et al. 2005) uses a supply-demand graph to represent the interdependency between infrastructures, and Markov and semi-Markov processes for describing the state transitions dynamics. Furthermore, the system performance is measured and analyzed using a probabilistic distribution. Reference (Faraji ... 

Limnios, N. Oprisan, G., 2005. Semi-Markov Processes and Reliability. Birkhauser Boston. 

We assume that the system during its operation process is taking v, Ke Y, different operation states z,Z2,..., z,. Eurther, we define the system operation process Z(t), te< 0,-too), with discrete operation states from the set (Z, Z2,..., z,. Moreover, we assume that the system operation process Z(t) is a semi-Markov process (Grabski 2002, Glynn Haas 2006, Ferreira Pacheco 2007, Limnios Oprisan 2001, Mercier 2008) with the conditional sojourn... 

Ferreira, E and Pacheco, A., 2007. Comparison of levelcrossing times for Markov and semi-Markov processes. Statistics Probability Letters, 77(2) 151-157. 

Glynn, P.W. and Haas P.J., 2006. Laws of large numbers and functional central limit theorems for generalized semi-Markov processes. Stochastic Models, 22(2) 201-231. 

The state-transition model can be analyzed using a number of approaches as a Markov chains, using semi-Markov processes or using Monte Carlo simulation (Fishman 1996). The applicability of each method depends on the assumptions that can be made regarding faults occurrence and a repair time. In case of the Markov approach, it is necessary to assume that both the faults and renewals occur with constant intensities (i.e. exponential distribution). Also the large number of states makes Markov or semi-Markov method more difficult to use. Presented in the previous section reliability model includes random values with exponential, truncated normal and discrete distributions as well as some periodic relations (staff working time), so it is hard to be solved by analytical methods. 

ABSTRACT The paper presents analytical and Monte Carlo simulation methods applied to the reliability evaluation of a complex multistate system. A semi-Markov process is applied to construct the multistate model of the system operation process and its main characteristics are determined. Analytical linking of the system operation process model with the system multistate reliability model is proposed to get a general reliability model of the complex system operating at varying in time operation conditions and to find its reliability characteristics. The application of Monte Carlo simulation based on the constructed general reliability model of the complex system is proposed to reliability evaluation of a port grain transportation system and the results of this application are illustrated and compared with the results obtained by analytical method. 

The relaxation of this exponentiality condition, which precisely coincides with the Markovian assumption, leads to semi-Markov processes (Feller, 1964). What could be the chemical conditions of the occurence of non-exponential waiting times, and which distributions could have physico -chemical relevance Though many works have been done with the method of the continuous time random walk (CTRW) mostly in connection with transport processes in amorphous media, it seems to be very hard job to associate chemical conditions to different kinds of waiting time distributions due to temporal disorders. ... 

From formal point of view it is a specific open question, whether which kinds of space and energy disorders can be converted to time disorders described by semi-Markov processes A semi-Markov process can be associated to a Markov process by identifying the expectation of the waiting time of the former with the waiting time of the latter. The stationary probability distribution of the two models is the same, but the realizations can be qualitatively different. 

Markov chains are random processes in which changes occur only at fixed times. However, many of the physical phenomena observed in everyday life are based on changes that occur continuously over time. Examples of these continuous processes are equipment breakdowns, arrival of telephone calls, and radioactive decay. Markov processes are random processes in which changes occur continuously over time, where the future depends only on the present state and is independent of history. This property provides the basic framework for investigations of system reliability, dependability, and safety. There are several different types of Markov processes. In a semi-Markov process, time between transitions is a random variable that depends on the transition. The discrete and continuous-time Markov processes are special cases of the semi-Markov process (as will be further explained). 

---