Linearizing data

Crowe, C.M., Recursive Identification of Gross Errors in Linear Data Reconciliation, AJChE Journal, 34(4), 1988,541-550. (Global chi square test, measurement test)... 

Figure 73. Projections of the concentration data onto each concentration factor vs. the corresponding projections of the spectral data onto each spectral factor for the noise-free, perfectly linear data.

Figure 2.20. Logarithmic transformations on x- ory-axes as used to linearize data. Notice how the confidence limits change in an asymmetric fashion. In the top row, the y-axis is transformed in the middle row, the x-axis is transformed in the bottom row, both axes are transformed simultaneously.

In practice, of course, this effect is very small, normally much smaller than any of the other sources of nonlinear behavior, and we are ordinarily safe in ignoring it, and calling Beer s law behavior linear in the absence of any of the other known sources of nonlinear behavior. However, the point here is that this completes the demonstration of our statement above, that Beer s law never exactly holds IN PRINCIPLE and that as spectroscopists we never ever really work with perfectly linear data. 

A minimum of 2 factors are required. The synthetic data did NOT demonstrate the advantage of a single linear wavelength over a multiple wavelength model, it merely illustrated the fact that a single linear factor is not sufficient to model non-linear data. We could stop here, but, for the sake of completeness-... 

In our original column on this topic [1] we had only done a principal component analysis to compare with the MLR results. One of the comments made, and it was made by all the responders, was to ask why we did not also do a PLS analysis of the synthetic linearity data. There were a number of reasons, and we offered to send the data to any or all of the responders who would care to do the PLS analysis and report the results. Of the original responders, Paul Chabot took us up on our offer. In addition, at the 1998 International Diffuse Reflectance Conference (The Chambersburg meeting), Susan Foulk also offered to do the PLS analysis of this data. 

Table 33-1 Summary of results obtained from synthetic linearity data using one PCA or PLS factor. We present only those performance results listed by the data analyst as Correlation Coefficient and Standard Error of Estimate...

Figure 64-1 A graphic illustration of the behavior of linear data. Figure 64-la - Linear data spread out around a straight line. Figure 64-lb - the residuals are spread evenly around zero.

As we expected, furthermore, for the normal , linear relationship, the /-value for the quadratic term for the linear data is not statistically significant. This demonstrates our contention that this method of testing linearity is indeed capable of distinguishing the two cases, in a manner that is statistically justifiable. 

Two situations arise in linear data reconciliation. Sometimes all the variables included in the process model are measured, but more frequently some variables are not measured. Both cases will be separately analyzed. 

Linear Data Reconciliation with All Measured Variables... 

The assumption that all variables are measured is usually not true, as in practice some of them are not measured and must be estimated. In the previous section the decomposition of the linear data reconciliation problem involving only measured variables was discussed, leading to a reduced least squares problem. In the following section,... 

Orthogonal factorizations may be applied to resolve problem (5.3) if the system of equations cp(x, u) = 0 is made up of linear mass balances and bilinear component and energy balances. After replacing the bilinear terms of the original model by the corresponding mass and energy flows, a linear data reconciliation problem results. 

The linear/linearized data reconciliation solution deserves some special attention because it allows the formulation of alternative strategies for the processing of information. In this chapter the mathematical formulation for the sequential processing of both constraints and measurements is analyzed. 

Linearized Data to be and measured or Data to be single linear/time... 

What is meant by perfectly linear data What value of what parameter would indicate that a data set is perfectly linear ... 

The term perfectly linear data refers to data in which the instrument readout is exactly the same multiple of the concentration at all concentrations measured and all the points lie exactly on the line. A value of exactly 1 for the correlation coefficient would indicate perfect linear data. 

Solver for non-linear data fitting tasks. Several examples are based on the fitting tasks already solved by the Newton-Gauss-Levenberg/Marquardt method in the earlier parts of this chapter. 

Further analysis of linearity data typically involves inspection of residuals for fit in the linear regression form and to verify that the distribution of data points around the line is random. Random distribution of residuals is ideal however, non-random patterns may exist. Depending on the distribution of the pattern seen in a plot of residuals, the results may uncover non-ideal conditions within the separation that may then help define the range of the method or indicate areas in which further development is required. An example of residual plot is shown in Figure 36. There was no apparent trend across injection linearity range. 

Nonlinear regression enables broad molecular weight distribution standards to be used for SBC calibration and permits simultaneous resolution correction euid calibration. Furthermore, it greatly increases the variety of equations 4iich can be fit to calibration curves, detector linearity data and chromatogreun shapes. 

Linearity Data from the regression line peaks are resolved from the peak of interest (e.g., Figure 2.4). Correlation... 

Linearity Data from regression line Correlation... 

Although ANNs can, in theory, model any relation between predictors and predictands, it was found that common regression methods such as PLS can outperform ANN solutions when linear or slightly nonlinear problems are considered [1-5]. In fact, although ANNs can model linear relationships, they require a long training time since a nonlinear technique is applied to linear data. Despite, ideally, for a perfectly linear and noise-free data set, the ANN performance tends asymptotically towards the linear model performance, in practical situations ANNs can reach a performance qualitatively similar to that of linear methods. Therefore, it seems not too reasonable to apply them before simpler alternatives have been considered. 

The system algorithm is the heart of the array device s ability to provide the needed information rapidly and in a useful format. The quantitative information can be easily obtained from the strongest linear data channel using traditional calibration techniques. More subtlety is needed to obtain the qualitative information described previously. How that is done is the subject of the following section. 

---