Random splitting

Cross validation and bootstrap techniques can be applied for a statistically based estimation of the optimum number of PCA components. The idea is to randomly split the data into training and test data. PCA is then applied to the training data and the observations from the test data are reconstmcted using 1 to m PCs. The prediction error to the real test data can be computed. Repeating this procedure many times indicates the distribution of the prediction errors when using 1 to m components, which then allows deciding on the optimal number of components. For more details see Section 3.7.1. 

CV or bootstrap is used to split the data set into different calibration sets and test sets. A calibration set is used as described above to create an optimized model and this is applied to the corresponding test set. All objects are principally used in training set, validation set, and test set however, an object is never simultaneously used for model creation and for test. This strategy (double CV or double bootstrap or a combination of CV and bootstrap) is applicable to a relatively small number of objects furthermore, the process can be repeated many times with different random splits resulting in a high number of test-set-predicted values (Section 4.2.5). 

A single CV as described gives n predictions. For many data sets in chemistry n is too small for a visualization of the error distribution. Furthermore, the obtained performance measure may heavily depend on the split of the objects into segments. It is therefore recommended to repeat the CV with different random splits into segments (repeated CV), and to summarize the results. Knowing the variability of MSEcv at different levels of model complexities also allows a better estimation of the optimum model complexity, see one standard error rule in Section 4.2.2 (Hastie et al. 2001). 

Different methods can be applied for the split into segments. Mode 111222333 denotes that the first n/s (rounded to integer) objects are put into segment 1, the next n/s objects into segment 2, and so on. Mode 123123123 puts object 1 into segment 1, object 2 into segment 2, and so on. Mode random makes a random split but usually without any user control. We recommend to sort the objects by a user-created random permutation of the numbers 1 to n, and then to apply mode 111222333 —and to repeat this several times. 

Also for double CV, the obtained performance measure depends on the split of the objects into segments, and therefore it is advisable to repeat the process with different random splits into segments (repeated double CV), and to summarize the results. 

Different selections of calibration and test data set may lead to different answers for the errors. In the following, we present results from one random split however, in the final overall comparison (Section 5.8.1.8) the evaluation scheme is repeated 100 times to get an idea of the distribution of the test error for the optimal parameter choice. 

Effect of Molecular Weight of Polyester on the Hydrolysis by Rhizopus lipase. Using three kinds of polyesters, PCL-diol (I), polyhexameth-ylene adipate (II), and a copolyester (ill) made from 1,6-hexamethyl-enediol and a 70 30 molar ratio mixture of e- caprolactone and adipic acid, the effects of the of polyester on the hydrolysis by lipase were examined (Figure k) Mn did not affect the rates of hydrolysis by R. arrhizus and delemar lipases when Vln was more than about UOOO. This would indicate these lipases randomly splits ester bonds in pol-mer chains. In contrast, when TEi was less than about i4000 2 the rates of the enzymatic hydrolysis were faster with the smaller Mn of polyesters. This corresponded to the fact that Tm was lower with the smaller Mn of polyesters. 

It may seem strange to see the normal distribution play a part in the p-value calculations in Section 11.5.1 and 11.5.2. The appearance of this distribution is in no sense related to the underlying distribution of the data. For the Mann-Whitney U-test for example it relates to the behaviour of the average of the ranks within each of the individual groups under the assumption of equal treatments where the ranks in those groups of sizes and 2 simply a random split of the numbers 1 through to Hi -b 2-... 

The concept of reducing the number of reaction vessels and exponentially increasing the number of synthesized compounds was brought to a next level of simplicity by the split-and-pool method of Furka et al.5 The split-and-pool method was independently applied by Lam et al.6 in a one-bead-one-compound concept for the combinatorial synthesis of large compound arrays (libraries) and by Houghten et al.7 for the iterative libraries. Now several millions peptides could be synthesized in a few days. In Furka s method the resin beads receiving the same amino acid were contained in one reaction vessel—identical to Frank s method—however, the beads were pooled and then split randomly before each combinatorial step. Thus the method is referred to as the random split-and-pool method to differentiate it from Frank s method in which each solid-phase particle was directed into a particular reaction vessel (the directed split-and-pool method). 

The distribution of compounds in a library is driven by statistical probabilities due to the random split process. Each compound is synthesized numerous times when the number of beads exceeds several times the number of compounds, or only a subset of compounds is produced when the number of beads is lower than the number of possible combinations of building blocks. 

The chemical history of the beads is lost. After each combinatorial step, the resin beads from all reaction vessels are pooled and randomly split into reaction vessels for the next combinatorial step. 

Fig. 3.13. Partial least-squares (PLS) calibration of the API data set (5 s accumulation time). Spectra were baseline corrected, normalised to unit length and mean centred. The data set was randomly split into a calibration set (two-thirds) and a prediction set (one-third) obvious outliers from the PCA analysis were excluded from the analysis. The graph shows predicted versus measured API concentration of the prediction set. The straight line represents the 45° diagonal (this figure was published in [65], Copyright Elsevier (2008))...

Two key questions in model selection are what proportion of molecules to use for a training set versus a test set when doing random splits of the data, and how many different training/test splits should be analyzed to obtain reliable inferences about model performance. The number of repetitions needed is surprisingly low and often the same decisions are made whether the number of training/test splits used was 200 or 20 or 10. 

The uniqueness of APLSl ( Randomizer ) is based on the fact that the 20 individual reaction compartments are connected into a larger continuous area. If there is a requirement to randomize (split and mix, or divide and recombine, in different author s terminology), the upper larger compartment is filled with the solvent, resin is pushed from the individual compartment into the common area by a flow of nitrogen, and the whole content is stirred. After sedimentation, the resin is uniformly distributed back into individual reaction chambers (95). Continuous stirring was used in the design of two synthesizers capable of resin randomization (85,96), one of which was commercialized but is now discontinued (82). 

A distribution of stresses in a crystal sample produces random splitting of the electronic levels and the net result is, to the first order, an inhomogeneous broadening of the electronic lines. 

Independent of these problems, we consider a rigorous assessment of the model quality as a fundamental issue. Many of the published models were not validated by an (external) test set. Even then, internal validation with LOO cross-validated tends to overestimate the predictive power of a model [133]. On the other hand, LGO cross-validation may suffer from a chance bias if only one random split is employed. External validation can not be afforded in many cases due to limited training samples available for model generation. 

Experimental outline completely random, split-plot in time (24 and 48 hours), 10 repetitions. Cultivation at Molasses (g/lOOmL of vinasse). Letters represent the groups formed by the Tukey test (5% confidence) the same letters = no significant variation. 

---