Data reconciliation

If the data consistency is acceptable, data can be reconciliated in order to remove the acceptable error [51]. The distance to the true estimate of the cumulated value is  

The same procedure is applicable to reconcilation of reaction rates using the error s. 

The main assumption in data reconciliation is that measurement values correspond to the steady state. However, process plants are rarely at steady state. Data reconciliation is used to manipulate the measured plant data to satisfy the steady-state assumption. Data reconciliation is used to detect instrument errors and leaks and to get smoother data for design calculations. 

If some measured variables are redundant, the solution of the balancing problem is based on reconciliation. In this case the measured values must be complemented by the information about their precision, which characterises random measurement errors, or the accuracy, which takes into account also the systematic part of overall error. 

Measurement error is defined as the difference between measured and true (usually unknown) value. The errors can be classified as 

Random errors are unpredictable errors whose values oscillate around zero. Their existence in the measuring process is inevitable. Random errors are governed by probabilistic laws and are fully described by the probability density function. The most important is so-called normal (Gauss) distribution which is fully characterised by the standard deviation (the mean value equals zero for random errors). The square of standard deviation is the variance of the measurement. The standard deviation o can be estimated from repeated measurement of one value  

Systematic errors have deterministic character. There are e.g. systematic errors in time, varying in some systematic manner (linear drift of zero of measurement instmments, etc.). Systematic errors can be often eliminated by the calibration of instruments, by the use of standards, etc. 

Gross errors (outliers) are large errors occurring from time to time as a result of inattention, measurement devices failures, unsteady state, etc. In data 

---

MacDonald, R.J. and C.S. Howat, Data Reconciliation and Parameter Estimation in Plant Performance Analysis, AlChE Journal, 34(1), 1988, 1-8. (Parameter estimation)... 

Crowe, C.M., Recursive Identification of Gross Errors in Linear Data Reconciliation, AJChE Journal, 34(4), 1988,541-550. (Global chi square test, measurement test)... 

Phillips, A.G. and D.P. Harrison, Gross Error Detection and Data Reconciliation in Experimental Kinetics, Indushial and Engineeiing Chemistiy Reseaieh, 32, 1993,2530-2536. (Measurement test)... 

Serth, R.W. and W.A. Heenan, Gross Error Detection and Data Reconciliation in Steam-Metering Systems, AlChE Journal, 32(5), 1986, 7.3.3-742. 

Leibovici, C.F., VS. Verneuil, Jr., and P. Yang, Improve Prediction with Data Reconciliation, Hydiocaihon Piocessing, October 199.3, 79-80. 

Some recent applications have benefited from advances in computing and computational techniques. Steady-state simulation is being used off-line for process analysis, design, and retrofit process simulators can model flow sheets with up to about a million equations by employing nested procedures. Other applications have resulted in great economic benefits these include on-line real-time optimization models for data reconciliation and parameter estimation followed by optimal adjustment of operating conditions. Models of up to 500,000 variables have been used on a refinery-wide basis. 

The use of this extended planning model will only be problematic if extra reference points, e.g., initial tank storage levels, have to be considered. This may lead to overdetermination of the model (i.e., conflicting level values for a given point in time) and it may be necessary to solve a data reconciliation problem. ... 

Soft sensors Fault detection Data reconciliation Statistical analysis Parameter estimation... 

Historically, treatment of measurement noise has been addressed through two distinct avenues. For steady-state data and processes, Kuehn and Davidson (1961) presented the seminal paper describing the data reconciliation problem based on least squares optimization. For dynamic data and processes, Kalman filtering (Gelb, 1974) has been successfully used to recursively smooth measurement data and estimate parameters. Both techniques were developed for linear systems and weighted least squares objective functions. 

The steady-state linear model data reconciliation problem can be stated as... 

Several researchers [e.g., Tjoa and Biegler (1992) and Robertson et al. (1996)] have demonstrated advantages of using nonlinear programming (NLP) techniques over such traditional data reconciliation methods as successive linearization for steady-state or dynamic processes. Through the inclusion of variable bounds and a more robust treatment of the nonlinear algebraic constraints, improved reconciliation performance can be realized. 

Extended Kalman filtering has been a popular method used in the literature to solve the dynamic data reconciliation problem (Muske and Edgar, 1998). As an alternative, the nonlinear dynamic data reconciliation problem with a weighted least squares objective function can be expressed as a moving horizon problem (Liebman et al., 1992), similar to that used for model predictive control discussed earlier. 

Liebman, M. J. T. F. Edgar and L. S. Lasdon. Efficient Data Reconciliation and Estimation for Dynamic Processes Using Nonlinear Programming Techniques. Comput Chem Eng 16(10/11) 963-986 (1992). 

Dynamic Data Reconciliation Using Nonlinear Programming Techniques 148... 

Joint Parameter Estimation-Data Reconciliation Problem 166... 

Principal Component Analysis in Data Reconciliation 219 11.5 Conclusions 223... 

Chemical process data inherently contain some degree of error, and this error may be random or systematic. Thus, the application of data reconciliation techniques allows optimal adjustment of measurement values to satisfy material and energy constraints. It also makes possible the estimation of unmeasured variables. It should be emphasized that, in today s highly competitive world market, resolving even small errors can lead to significant improvements in plant performance and economy. This book attempts to provide a comprehensive statement, analysis, and discussion of the main issues that emerge in the treatment and reconciliation of plant data. 

All of the previous ideas are developed further in Chapter 8, where the analysis of dynamic and quasi-steady-state processes is considered. Chapter 9 is devoted to the general problem of joint parameter estimation-data reconciliation, an important issue in assessing plant performance. In addition, some techniques for estimating the covariance matrix from the measurements are discussed in Chapter 10. New trends in this field are summarized in Chapter 11, and the last chapter is devoted to illustrations of the application of the previously presented techniques to various practical cases. 

With the advance of computer techniques, especially implementation of distributed control systems (DCS) to chemical processes, a large set of on-line measurements are available at every sampling period. The rational use of this large volume of data requires the application of suitable techniques to improve their accuracy. This goal has triggered the focus on research and development, during the last ten years, in the area of plant data reconciliation. Complete reviews on the subject can be found in the works of Mah (1990), Madron (1992), and Crowe (1996). 

FIGURE 3 Typical arrangement between the DCS and the Data Reconciliation, Simulation, and Optimization procedures (from Simulation Sciences, Inc., 1989). 

A simplified diagram of the general procedure for data reconciliation in chemical plants is given in Fig. 4. 

SOME ISSUES ASSOCIATED WITH A GENERAL DATA RECONCILIATION PROBLEM... 

The presence of gross errors invalidates the statistical basis of the common data reconciliation procedures, so they must be identified and removed. Gross error detection has received considerable attention in the past 20 years. Statistical tests in combination with an identification strategy have been used for this purpose. A good survey of the available methodologies can be found in Mah (1990) and Crowe (1996). 

Finally, approaches are emerging within the data reconciliation problem, such as Bayesian approaches and robust estimation techniques, as well as strategies that use Principal Component Analysis. They offer viable alternatives to traditional methods and provide new grounds for further improvement. 

It is our goal in this book to address the problems, introduced earlier, that arise in a general data reconciliation problem. It is the culmination of several years of research and implementation of data reconciliation aspects in Argentina, the United States, and Australia. It is designed to provide a simple, smooth, and readable account of all aspects involved in data classification and reconciliation, while providing the interested reader with material, problems, and directions for further study. 

Chapter 5 deals with steady-state data reconciliation problem, from both a linear and a nonlinear point of view. Special consideration is given, in Chapter 6, to the problem of sequential processing of information. This has several advantages when compared with classical batch processing. 

Chapter 9 deals with the general problem of joint parameter estimation data reconciliation. Starting from the typical parameter estimation problem, the more general formulation in terms of the error-in-variable methods is described, where measurement errors in all variables are considered. Some solution techniques are also described here. 

---