VALIDATE technique

Before a procedure can provide useful analytical information, it is necessary to demonstrate that it is capable of providing acceptable results. Validation is an evaluation of whether the precision and accuracy obtained by following the procedure are appropriate for the problem. In addition, validation ensures that the written procedure has sufficient detail so that different analysts or laboratories following the same procedure obtain comparable results. Ideally, validation uses a standard sample whose composition closely matches the samples for which the procedure was developed. The comparison of replicate analyses can be used to evaluate the procedure s precision and accuracy. Intralaboratory and interlaboratory differences in the procedure also can be evaluated. In the absence of appropriate standards, accuracy can be evaluated by comparing results obtained with a new method to those obtained using a method of known accuracy. Chapter 14 provides a more detailed discussion of validation techniques. 

The number of latent variables (PLS components) must be determined by some sort of validation technique, e.g., cross-validation [42], The PLS solution will coincide with the corresponding MLR solution when the number of latent variables becomes equal to the number of descriptors used in the analysis. The validation technique, at the same time, also serves the purpose to avoid overfitting of the model. 

It is not easy to determine whether lumping in a process model is a valid technique for representing the process. A good rule of thumb is that if the response is... 

It is important to note that theoretic argument and empiric study have shown that the LOO cross-validation approach is preferred to the use of an external test set for small to moderate sized chemical databases [39]. The problems with holding out an external test set include (1) structural features of the held out chemicals are not included in the modeling process, resulting in a loss of information, (2) predictions are made only on a subset of the available compounds, whereas LOO predicts the activity value for all compounds, and (3) personal bias can easily be introduced in selection of the external test set. The reader is referred to Hawkins et al. [39] and Kraker et al. [40] in addition to Section 31.6 for further discussion of proper model validation techniques. 

Like MLR, however, one must be careful to avoid the temptation of overfitting the PCR model. In this case, overfitting can occur through the use of too many principal components, thus adding unwanted noise to the model and making the model more sensitive to unforeseen disturbances. Model validation techniques (discussed in Section 12.4) can be used to avoid overfitting of PCR models. 

In PAT, one is often faced with the task of building, optimizing, evaluating, and deploying a model based on a limited set of calibration data. In such a situation, one can use model validation and cross-validation techniques to perform two of these functions namely to optimize the model by determining the optimal model complexity and to perform preliminary evaluation of the model s performance before it is deployed. There are several validation methods that are commonly used in PAT applications, and some of these are discussed below. 

It should be mentioned that another validation technique, called leverage correction [1], is available in some software packages. This method, unlike cross validation, does not involve splitting of the calibration data into model and test sets, but is simply an altered calculation of the RMSEE fit error of a model. This alteration involves the weighting of the contribution of the root mean square error from each calibration... 

NIR models are validated in order to ensure quality in the analytical results obtained in applying the method developed to samples independent of those used in the calibration process. Although constructing the model involves the use of validation techniques that allow some basic characteristics of the model to be established, a set of samples not employed in the calibration process is required for prediction in order to conhrm the goodness of the model. Such samples can be selected from the initial set, and should possess the same properties as those in the calibration set. The quality of the results is assessed in terms of parameters such as the relative standard error of prediction (RSEP) or the root mean square error of prediction (RMSEP). 

Since the point source estimated incident heat flux is much greater than 5 kW/m, the limit for applicability, it is clear that the people are too close to the fire for the point source to be a valid technique. Therefore, an estimation using the method of Shokri and Beyler is warranted. The emissive power of the fire is calculated [Equation (5-12)] as ... 

In this example, two principal components are arbitrarily selected. More or fewer may be necessary, and this is a function of a predetermined stopping rule for extraction of principal components from X. In SIMCA method, a cross validation technique (2) is used. 

There is no recommendation of one of the introduced methods (MIP, BET, or ISEC) as the most accurate, reliable, and universally valid technique for the determination of the porous properties of a stationary phase. MIP, BET, and ISEC have rather to be regarded as three independent methodologies, those results complement one another to yield a precise estimation of the porosity of an investigated column packing. The most important characteristics, limitations, and methodological strengths of MIP, BET, and ISEC are intended to be discussed in this section. 

The software requires the following information the concentration and spectral data, the preprocessing selections, the maximum number of factors to estimate, and the validation approach used to choose the optimal number of factors. The maximum rank selected is 10 for constructing the model to predict the caustic concentration. The validation technique is leave-one-out cross-validation where an entire design point is left out. Tliat is, there are 12 cross validation steps and all spectra for each standard (at various temperatures) are left out of the model building phase at each step. 

Figarole PL. Computer software validation techniques. DIA Conference on Computer Validation, Jan. 21-23, 1985. 

Describes the validation technique retrospective, concurrent, or prospective ... 

The liquid-liquid, acid-base fractionation method was preferred over HPLC by the liquid waste panel members because it is a better validated technique for the isolation of components that are toxic or that might otherwise interfere with the assay. Also, the distribution of activity in the acid, base, and neutral fractions provides a preliminary estimate of the types of chemicals responsible for the mutagenicity. 

Another potential disadvantage of PLS over PCR is that there is a higher potential to overfit the model through the use of too many PLS factors, especially if the Y-data are rather noisy. In such situations, one could easily run into a situation where the addition of a PLS factor helps to explain noise in the Y-data, thus improving the model fit without an improvement in real predictive ability. As for all other quantitative regression methods, the use of validation techniques is critical to avoid model overfitting (see Section 8.3.7). 

If overfitting and under-fitting are such big problems, then how can one avoid them The most commonly used tools for combating them are called model validation techniques. There are several tools that fall under this category, but they all operate with the same objective attempt to assess the performance of the model when it is applied to data that were not used to build it ... 

In external validation, a model is tested using data that were not used to build the model. This type of validation is the most intuitively straightforward of the validation techniques. If the external samples are sufficiently representative of the samples that will be applied to the model during its operation, then this technique can be used to provide a reasonable assessment of the model s prediction performance on future samples, as well as to provide a good assessment of the optimal complexity of the model. 

In contrast to external validation, internal validation involves the use of the calibration data only, and does not require the collection and preparation of additional validation samples. There are several different techniques that are considered internal validation techniques, and these are mentioned in the following sections. 

The principal component space has several advantages over selected X-variable spaces. Concerns about redundancy are eliminated because the PCs are orthogonal to one another. In addition, because each PC explains the most remaining variance in the X-data, it is often the case that fewer PCs than original X-variables are needed to capture the same information in the X-data. This leads to simpler classification models, less susceptibility to overfitting through the use of too many dimensions in the model space, and less noise in the model. Nonetheless, it is still a good idea to use validation techniques even when PCs are used to define the space. 

Integration is a process of determining the area under a chromatographic peak for the quantitation of the analyte concentration. There are several scientifically valid techniques that may be used to integrate peaks. Modern chromatography computer software performs peak integration with a minimum of human intervention. Once an... 

---