Data preprocessing/scaling

The SIMCA method has been developed to overcome some of these limitations. The SIMCA model consists of a collection of PCA models with one for each class in the dataset. This is shown graphically in Figure 10. The four graphs show one model for each excipient. Note that these score plots have their origin at the center of the dataset, and the blue dashed line marks the 95% confidence limit calculated based upon the variability of the data. To use the SIMCA method, a PCA model is built for each class. These class models are built to optimize the description of a particular excipient. Thus, each model contains all the usual parts of a PCA model mean vector, scaling information, data preprocessing, etc., and they can have a different number of PCs, i.e., the number of PCs should be appropriate for the class dataset. In other words, each model is a fully independent PCA model. 

Undoubtedly, however, the appearance and interpretation not only of PC plots but also of almost all chemometric techniques, depend on data preprocessing. The influence of preprocessing can be dramatic, so it is essential for the user of chemometric software to understand and question how and why the data have been scaled prior to interpreting die result from a package. More consequences are described in Chapter 6. 

There are many educational and commercial software packages available for development and deployment of ANNs. Some of those packages such as Gensym s NeurOn-Line Studio include data preprocessing modules to filter or scale data and eliminate outliers [89]. 

Data preprocessing is a very important step in many chemometric techniques, which can be used separately (before a method is applied), or as a self-adjusting procedure that forms part of the chemometric methods. Ideally, data preprocessing can be used to remove known interference(s) from data to improve selectivity and enhance more important information to improve robustness. Techniques such as principal components (PCs see Principal components regression section below) are scale dependent. If one variate has a much higher variance than the others, it is necessary to scale the original variates before... 

Rarely will your raw data be in an acceptable form for input to an ANN. You must perform some combination of transforming and scaling to both the input and output data. Preprocessing is often the single most important operation in the development of a successful application, and you are enthusiastically urged to pay close attention to it. Time spent here will more than repay itself later. Remember garbage in, garbage out. 

What type of data analysis will be needed It could be simple band area or band ratio analyses, curve-fitting analysis, or sophisticated chemometric approaches. In this step, data preprocessing steps can be identified and implemented for baseline correction, scaling and so forth, (see Chapter 7 for a further discussion on this subject.)... 

When the data set X5 is loaded into the workspace (using load X5), go to File , select Load Data , load X-block and load X5. Data auto-scaling should be performed You could select Preprocess to change this. Now calculate the principal components by selecting calc , the window as shown in Fig. 22.3 will appear. 

Data preprocessing. The data should preferably be evenly distributed over the entire operating range of the process. If the process data are noisy, the noise should preferably be removed by using an appropriate smoothing or filtering technique. In addition, outhers should be removed, since they would affect the accuracy and prediction capability of the model. Data normalization. Process values usually take arbitrary values and it can be expected that they will not all be of the same magnitude. It is therefore recommended to scale all process values between 0.1 and 0.9 to avoid saturation of the hidden nodes and to ensme that all process variables have an impact on the output. 

In data analysis, data are seldom used without some preprocessing. Such preprocessing is typically concerned with the scale of data. In this regard two main scaling procedures are widely used zero-centered and autoscaling. 

The fundamental elements of deterministic models involve a combination of chemical and meteorologic input, preprocessing with data transmission, logic that describes atmospheric processes, and concentration-field output tables or displays. In addition to deterministic models, there are statistical schemes that relate precursors (or emission) to photo-chemical-oxidant concentrations. Models may be classified according to time and space scales, depending on the purposes for which th are designed. 

GLS preprocessing can be considered a more elaborate form of variable scaling, where, instead of each variable having its own scaling factor (as in autoscaling and variable-specific scaling), the variables are scaled to de-emphasize multivariate directions that are known to correspond to irrelevant spectral effects. Of course, the effectiveness of GLS depends on the ability to collect data that can be used to determine the difference effects, the accuracy of the measured difference effects, and whether the irrelevant spectral information can be accurately expressed as linear combinations of the original x variables. 

The graphical representation that is universally available is shown in Figure 4.14. This plot is examined for features such as sample outliers, the need for preprocessing, questionable variables, and other patterns. In this case, there appears to be three clusters and n o additional "unusual" samples. These two samples are within the scale of the entire data set. but have a different response pattern. 

Although the shape of the Visual-Empirical Region-of-Influence (VERI) mask is invariable, it size scales automatically according to the properties of the cluster (Fig. 10.10b). The VERI algorithm requires preprocessing of the data and for that purpose PCA or PCR preprocessing is routinely used. It is relatively immune to the presence of unknowns, and nonlinearity and nonadditivity of sensor responses (Osbourn et al., 1998). It has been used successfully to determine the optimum... 

In some cases standardisation (or closely related scaling) is an essential first step in data analysis. In case study 2, each type of chromatographic measurement is on a different scale. For example, the N values may exceed 10 000, whereas k rarely exceeds 2. If these two types of information were not standardised, PCA will be dominated primarily by changes in N, hence all analysis of case study 2 in this chapter involves preprocessing via standardisation. Standardisation is also useful in areas such as quantitative structure-property relationships, where many different pieces of information are measured on very different scales, such as bond lengths and dipoles. 

---