First-order systems discrete-time model

Figure 17.8 shows the effect of pole location on the possible responses for a simple first-order transfer function, G(z) = bo/(l - az )y forced by an impulse at A = 0. The corresponding continuous-time model responses are also shown. Poles 3 and 4 are inside the unit circle and thus are stable, while poles 1 and 6 are outside the unit circle and cause an unstable response. Poles 2 and 5 lie on the unit circle and are marginally stable. Negative poles such as 4-6 produce oscillatory responses, even for a first-order discrete-time system, in contrast to continuous-time first-order systems. 

The monomolecular reaction systems of chemical kinetics are examples of linear coupled systems. Since linear coupled systems are the simplest systems with many degrees of freedom, their importance extends far beyond chemical kinetics. The linear coupled systems in which we are interested may be characterized, in general terms, as arising from stochastic or Markov processes that are continuous in time and discrete in an appropriate space. In addition, the principle of detailed balancing is observed and the total amount of material in the system is conserved. The system is characterized by discrete compartments or states and material passes between these compartments by first order processes. Such linear systems are good models for a large number of processes. 

A common approach is to model a packed bed reactor as a system of partial differential equations and in order to investigate the behaviour of such a reactor numerically, these model equations have to be discretized. For all four methods discussed in this section, first the space variable of the dimensionless equations is discretized. This leads to a large system of N (where JV depends on the space discretization) time periodic ordinary differential equations. This system can be written as... 

Nassar et al. [10] employed a stochastic approach, namely a Markov process with transient and absorbing states, to model in a unified fashion both complex linear first-order chemical reactions, involving molecules of multiple types, and mixing, accompanied by flow in an nonsteady- or steady-state continuous-flow reactor. Chou et al. [11] extended this system with nonlinear chemical reactions by means of Markov chains. An assumption is made that transitiions occur instantaneously at each instant of the discretized time. 

As mentioned before, the Laplace transform is a convenient way of representing a process model. However, in today s world of computers discrete time representations of systems is preferred. The simplest way is a first-order discretization ... 

In order to give a good description of the problem, we shall model it as a Markov Decision Problem (MDP). Markov Decision Processes have been studied initially by Bellmann (1957) and Howard (1960). We will first give a short description of an MDP in general. Suppose a system is observed at discrete points of time. At each time point the system may be in one of a finite number of states, labeled by 1,2,.., M. If, at time t, the system is in state i, one may choose an action a, fix>m a finite space A. This action results in a probability PJ- of finding the system in state j at time r+1. Furthermore costs qf have to be paid when in state i action a is taken. 

---