Validation set

When network weights have been trained to appropriate values, the NSC is ready to start classifying. The data set to be classified is specified in the same manner as previously used for the training and validation sets. The classifier is applied to data through the use of a menu and generates a list including filenames, suggested class and the neuron outputs from the output layer (used for decision). The result is currently presented in a simple text editor, from which it can be saved and included in other documents. 

A solvent free, fast and environmentally friendly near infrared-based methodology was developed for the determination and quality control of 11 pesticides in commercially available formulations. This methodology was based on the direct measurement of the diffuse reflectance spectra of solid samples inside glass vials and a multivariate calibration model to determine the active principle concentration in agrochemicals. The proposed PLS model was made using 11 known commercial and 22 doped samples (11 under and 11 over dosed) for calibration and 22 different formulations as the validation set. For Buprofezin, Chlorsulfuron, Cyromazine, Daminozide, Diuron and Iprodione determination, the information in the spectral range between 1618 and 2630 nm of the reflectance spectra was employed. On the other hand, for Bensulfuron, Fenoxycarb, Metalaxyl, Procymidone and Tricyclazole determination, the first order derivative spectra in the range between 1618 and 2630 nm was used. In both cases, a linear remove correction was applied. Mean accuracy errors between 0.5 and 3.1% were obtained for the validation set. 

Pitot-static traverse The set positions of a Prandtl tube m a duct run required to provide a statistically valid set of readings, A series of measurements of the torn 1 and static pressure taken across an area of a duct to determine the air veloc-it> at that point. The sampling distance should be at least 7.5 times the diameter of the duct away from any disturbances of air flow. 

Which ones could not occur For the valid sets, identify the orbital involved. 

Strategy Use the selection rules for the four quantum numbers to find the sets that could not occur. For the valid sets, identify the principal level and sublevel... 

The data in the validation set are used to challenge the calibration. We treat the validation samples as if they are unknowns. We use the calibration developed with the training set to predict (or estimate) the concentrations of the components in the validation samples. We then compare these predicted concentrations to the actual concentrations as determined by an independent referee method (these are also called the expected concentrations). In this way, we can assess the expected performance of the calibration on actual unknowns. To the extent that the validation samples are a good representation of all the unknown samples we will encounter, this validation step will provide a reliable estimate of the calibration s performance on the unknowns. But if we encounter unknowns that are significantly different from the validation samples, we are likely to be surprised by the actual performance of the calibration (and such surprises are seldom pleasant). 

We often cannot afford to assemble large numbers of validations samples with concentrations as accurate as the training set concentrations. But since the validation samples are used to test the calibration rather than produce the calibration, errors in validation sample concentrations do not have the same detrimental impact as errors in the training set concentrations. Validation set... 

Generally speaking, the more validation samples the better. It is nice to have at least as many samples in the validation set as were needed in the training set. It is even better to have considerably more validation samples than calibration samples. 

We will organize our data into training sets and validation sets. The training sets will be used to develop the various calibrations, and the validation sets will be used to evaluate how well the calibrations perform. 

Next, we create a concentration matrix containing mixtures that we will hold in reserve as validation data. We will assemble 10 different validation samples into a concentration matrix called C3. Each of the samples in this validation set will have a random amount of each component determined by choosing numbers randomly from a uniform distribution of random numbers between 0 and 1. 

We will create yet another set of validation data containing samples that have an additional component that was not present in any of the calibration samples. This will allow us to observe what happens when we try to use a calibration to predict the concentrations of an unknown that contains an unexpected interferent. We will assemble 8 of these samples into a concentration matrix called C5. The concentration value for each of the components in each sample will be chosen randomly from a uniform distribution of random numbers between 0 and I. Figure 9 contains multivariate plots of the first three components of the validation sets. 

Figure 9. Concentration values for first 3 components of the validation sets.

We can see, in Figure 20 that we get similar results when we use the two calibrations, Kl , and K2(al, to predict the concentrations in the validation sets. When we examine the plots for K13m and K23w die predictions for our normal validation set, A3, we see that, while the calibrations do work to a certain degree, there is a considerable amount of scatter between the expected and the predicted values. For some applications, this might be an acceptable level of performance. But, in general, we would hope to do much better. 

Figure 23 contains plots of the expected vs. predicted concentrations for all of the nonzero intercept CLS results. We can easily see that these results are much better than the results of the first calibrations. It is also apparent that when we predict the concentrations from the spectra in A5, the validation set with the... 

It would be interesting to see how well CLS would have done if we hadn t had a component whose concentration values were unknown (Component 4). To explore this, we will create two more data sets, A6, and A7, which will not contain Component 4. Other than the elimination of the 4randomly structured training set, and A7 will be identical to A3, the normal validation set. The noise levels in A6, A7, and their corresponding concentration matrices, C6 and C7, will be the same as in A2, A3, C2, and C3. But, the actual noise will be newly created—it won t be the exact same noise. The amount of nonlinearity will be the same, but since we will not have any absorbances from the 4 component, the impact of the nonlinearity will be slightly less. Figure 24 contains plots of the spectra in A6 and A7. 

Cross-validation. We don t always have a sufficient set of independent validation samples with which to calculate PRESS. In such instances, we can use the original training set to simulate a validation set. This approach is called cross-validation. The most commom form of cross-validation is performed as follows ... 

Now, we are ready to apply PCR to our simulated data set. For each training set absorbance matrix, A1 and A2, we will find all of the possible eigenvectors. Then, we will decide how many to keep as our basis set. Next, we will construct calibrations by using ILS in the new coordinate system defined by the basis set. Finally, we will use the calibrations to predict the concentrations for our validation sets. 

If we did not have a validation set available to us, we could use cross-validation for the same purposes. Figure 55 contains plots of the results of cross validation of the two training sets, A1 and A2. Since no separate validation data set is involved, we name the results PCRCROSS1 and PCRCROSS2, respectively. 

Figure 62. Plots (left column) of a regenerated spectrum ( — ) and the original spectrum ( ) of a sample from each of the 3 validation sets A3, A4, and A5. Residuals for all of the regenerated spectra in each of the 3 validation sets are also shown (right column).

Table 12 shows the SSR s for each sample in validation set AS together with the concentration of the unexpected component in each sample. Figure 63 contains a plot of the data in Table 12. 

We can see, in Table 12 that there is a monotonic relationship between the SSR and the concentration of the interferring 5,h component in each sample of the validation set A5. In Figure 63 we can see that the relationship is approximately linear with the square root of the SSR. The important thing is not the linearity of the relationship, but that it exists at all and increases monotonically. It gives us a very useful way of flagging samples which our... 

Table 12. Sum of the square of residuals (SSR) for the individual samples in the validation set, AS, using the S basis vectors for Al together with the concentrations of the unexpected 5th component in the A5 samples.

---