The state space

Deflnition. The state space SS of a Markov chain is the set of all states a system can occupy. It is designated by  

S designates state where the subscript stands for the number of the state. The states are exclusive of one another, that is, no two states can occur or be occupied simultaneously. This point is clearly elaborated in example 2.9 in the following and its opposite in Fig.2-1 with the associated explanations. Markov chains are applicable only to systems where the number of states Z is finite or countably infinite. In the latter case, an infinite number of states can be arranged in 

Note that Si must be occupied before occupying each of the others. It should be noted that Eqs.(2-12a) and (2-12b) satisfy  

The summation on the left-hand side designates a state (or event) comprised of Z fundamental states. The prob summation means the probability of occupying at least one of the states [Si, S2, S3. Sz]. 

Dennition. A Markov chain is said to be a finite Markov chain if the state space is finite. 

---

The state-space approach is a generalized time-domain method for modelling, analysing and designing a wide range of control systems and is particularly well suited to digital computational techniques. The approach can deal with... 

The state variables are the smallest number of states that are required to describe the dynamic nature of the system, and it is not a necessary constraint that they are measurable. The manner in which the state variables change as a function of time may be thought of as a trajectory in n dimensional space, called the state-space. Two-dimensional state-space is sometimes referred to as the phase-plane when one state is the derivative of the other. 

Coupled-map Lattices. Another obvious generalization is to lift the restriction that sites can take on only one of a few discrete values. Coupled-map lattices are CA models in which continuity is restored to the state space. That is to say, the cell values are no longer constrained to take on only the values 0 and 1 as in the examples discussed above, but can now take on arbitrary real values. First introduced by Kaneko [kaneko83]-[kaneko93], such systems are simpler than partial differential equations but more complex than generic CA. Coupled-map lattices are discussed in chapter 8. 

Coupled-Map Lattices these are models in which continuity is restored to the state space ... 

One obvious way of making the model more amenable to a CA implementation is to permanently attach the demons to each site. In order to accommodate such demons, the state space of each site must also be increased from two to four bits ... 

The general implications of the sufficient theory (i.e., the state-space sufficient theory). 

The way we have stated the domain theory for the state-space representation has enabled us to avoid making explicit reference to the alphabet symbol properties. However, if in other formulations we need to refer to these properties, we would again use a recursive parsing of the list of symbols to enable generalization over the size of the alphabet. 

Equations (41.15) and (41.19) for the extrapolation and update of system states form the so-called state-space model. The solution of the state-space model has been derived by Kalman and is known as the Kalman filter. Assumptions are that the measurement noise v(j) and the system noise w(/) are random and independent, normally distributed, white and uncorrelated. This leads to the general formulation of a Kalman filter given in Table 41.10. Equations (41.15) and (41.19) account for the time dependence of the system. Eq. (41.15) is the system equation which tells us how the system behaves in time (here in j units). Equation (41.16) expresses how the uncertainty in the system state grows as a function of time (here in j units) if no observations would be made. Q(j - 1) is the variance-covariance matrix of the system noise which contains the variance of w. 

Example 4.1 Derive the state space representation of a second order differential equation of a form similar to Eq. (3-16) on page 3-5 ... 

Example 4.2 Draw the block diagram of the state space u ... 

X Example 4.3 Let s try another model with a slightly more complex input. Derive the state space representation of the differential equation... 

Example 4.5 Derive the state space representation of two continuous flow stirred-tank reactors in series (CSTR-in-series). Chemical reaction is first order in both reactors. The reactor volumes are fixed, but the volumetric flow rate and inlet concentration are functions of time. 

To derive the state space representation, one visual approach is to identify locations in the block diagram where we can assign state variables and write out the individual transfer functions. In this example, we have chosen to use (Fig. E4.6)... 

We can check with MATLAB that the model matrix A has eigenvalues -0.29, -0.69, and -10.02. They are identical with the closed-loop poles. Given a block diagram, MATLAB can put the state space model together for us easily. To do that, we need to learn some closed-loop MATLAB functions, and we will defer this illustration to MATLAB Session 5. 

An important reminder Eq. (E4-23) has zero initial conditions x(0) = 0. This is a direct consequence of deriving the state space representation from transfer functions. Otherwise, Eq. (4-1) is not subjected to this restriction. 

With the state space model, substitution of numerical values in (E4-18) leads to the dynamic equations... 

To complete the state space model, the output equation is... 

Until we can substitute numerical values and turn the problem over to a computer, we have to admit that the state space form in (E4-47) is much cleaner to work with. 

A tool that we should be familiar with from introductory linear algebra is similarity transform, which allows us to transform a matrix into another one but which retains the same eigenvalues. If a state x and another x are related via a so-called similarity transformation, the state space representations constmcted with x and x are considered to be equivalent.1... 

S is the state space system Default is the diagonal form This is the observable companion... 

When we used root locus for controller design in Chapter 7, we chose a dominant pole (or a conjugate pair if complex). With state space representation, we have the mathematical tool to choose all the closed-loop poles. To begin, we restate the state space model in Eqs. (4-1) and (4-2) ... 

The state space state feedback gain (K2) related to the output variable C2 is the same as the proportional gain obtained with root locus. Given any set of closed-loop poles, we can find the state feedback gain of a controllable system using state-space pole placement methods. The use of root locus is not necessary, but it is a handy tool that we can take advantage of. 

Make the state space object from % the transfer function %Rescale MATLAB model matrices... 

MATLAB will return the vector [0 1.29], meaning that K, = 0, and K2 = 1.29, which was the proportional gain obtained in Example 7.5A. Since K, = 0, we only feedback the controlled variable as analogous to proportional control. In this very simple example, the state space system is virtually the classical system with a proportional controller. 

A note of caution is necessary when we let MATLAB generate the state space model from a transfer function. The vector C (from S. c) is [0 0.5], which means that the indexing is reversed such that x2 is the output variable, and xl is the derivative of x2. Secondly, C is not [0 1], and hence we have to rescale the matrices B and C. These two points are further covered in MALTA Session 4. 

This differential equation is used to compute xel, which then is used to calculate xe with (9-50). With the estimated states, we can compute the feedback to the state space model as... 

Do the time response simulation in Example 7.5B. We found that the state space system has a steady state error. Implement integral control and find the new state feedback gain vector. Perform a time response simulation to confirm the result. 

What we want is to dictate that the transfer function of this estimator is the same as that of the state space model ... 

We should see that the LTI object is identical to the state space model. We can retrieve and operate on individual properties of an object. For example, to find the eigenvalues of the matrix a inside sys obj ... 

Now, you may wonder if we can generate the state space model directly from a transfer function. The answer is, of course, yes. We can use... 

The third alternative to generate the diagonalized form is to use the state space to state space transformation function. The transformation is based on the modal matrix that we obtained earlier. 

We now repeat the same ideas one more time with Example 4.9. We first make the transfer function and the state space objects ... 

If this is not enough to convince you that everything is consistent, try step o on the transfer function and different forms of the state space model. You should see the same unit step response. 

To test an operation refinement, you push a variety of signals in, from different initial states, and see whether the right responses come out. There are various ways to use a precise specification to generate test cases that reasonably cover the state space of input parameters and initial state. 

The state-space description of a process is given below ... 

---