Bootstrap sampling

Fig. 1. Unrooted phylogenetic tree based on the core amino acid sequences of 113 catalases. The numbers at the three main nodes represent the proportion (out of 100) of bootstrap sampling that supports the topology. The three main clades are circled for clarity.

Bootstrap sample A sample (e.g., 5000) obtained from an original data set by randomly drawing, with replacement, 5000 values from the original sample or a distribution estimated for that sample. 

Bootstrapping involves the repetitive drawing of random samples with replacement from the observed population and computing statistics. A complete bootstrap in an observed population with eight variables would require the calculation of bootstrap statistics for 8 = 16,777,216 samples, quite a computer-intensive process. Therefore bootstrap samples are usually Umited to hundreds or thousands of drawings. 

RF [29,30] is an ensemble of unpruned classification trees separately grown from bootstrap samples of the training data set. A subset of nitry input variables is randomly selected as candidates to determine the best possible split at each node during the tree induction. The final prediction is generally made by aggregating the outputs of all the ntree trees generated in the forest. The unbiased out-of-bag (OOB) estimate of the generalization error is used to internally evaluate the prediction performance ofRF. 

Step 6. With the appropriate pharmacostatistical models, population model building is performed using covariates retained in step 5 with the covariate selection level set at a= 0.005. The backward elimination for covariate selection in applied to each of the 100 bootstrap samples. The covariates found to be important in explaining the variablilty in the parameter of interest are used to build the final population PM model. 

One hundred bootstrap samples are generated and the appropriate structural model that best describes the data from each sample is determined. This is done to ensure that the model that best describes the bootstrap data is not different from the basic structural model used for developing the population PK model for the data before bootstrapping. With the right structural model POSTHOC individual Bayesian estimates are generated and the data subjected to GAM. 

Step 2. Generate 100 bootstrap samples, each having the same sample size as the original data set, using nonparametric bootstrap. 

A bootstrap sample is generated by repeated random sampling, with replacement, of an m-sized pseudosample from the original data set. At each sampling step, every vector x, has an equal probability of being chosen. Thus, for a given iteration, it is possible to choose three of Xi, none of X2, five of X3, and so forth. 

This sampling is repeated until the bootstrap sample also consists of m vectors, Y = (xf, xf,..., xf,..., xt, G, where the vector xf represents all the observations for the ith randomly selected subject. 

The smoothed bootstrap has been proposed to deal with the discreteness of the empirical distribution function (F) when there are small sample sizes (A < 15). For this approach one must smooth the empirical distribution function and then bootstrap samples are drawn from the smoothed empirical distribution function, for example, from a kernel density estimate. However, it is evident that the proper selection of the smoothing parameter (h) is important so that oversmoothing or undersmoothing does not occur. It is difficult to know the most appropriate value for h and once the value for h is assigned it influences the variability and thus makes characterizing the variability terms of the model impossible. There are few studies where the smoothed bootstrap has been applied (21,27,28). In one such study the improvement in the correlation coefficient when compared to the standard non-parametric bootstrap was modest (21). Therefore, the value and behavior of the smoothed bootstrap are not clear. 

Select B independent bootstrap samples. This will usually be at least 100 bootstrap data sets for a PM model (1). 

Perform the evaluation of interest on each bootstrap sample, estimating the parameter of interest from each sample. 

Estimate the SE of the parameter of interest by the sample standard deviation from B bootstrap samples ... 

From the original data set (Do), 505 bootstrap data sets were constructed by resampling with replacement. The sampling was repeated until each bootstrap sample consisted of N subjects, where in this case N = 323. (See Table 15.1 for an extensive explanation of the notation that follows.) These 505 bootstrap data sets were denoted as Di to Dsos. The structure (So) of the model Mo (Mo = F(So)) was retained. What was meant by retaining the structure for Mo was that the coefficients that related the PK parameters (i.e., clearance and apparent volume) to covariates (i.e., weight) were allowed to be estimated from each of the 505 bootstrap data sets. So that for this study, the following was the structural model (So) ... 

D. Bootstrapped data or samples (had 323 subjects (i = 1, 2, 3,..., 323) subjects data) (Do), on which the developed population pharmacokinetic model was based) were drawn with replacement from the observed data (Do) observed data could either appear in the bootstrap samples (D,) once, more than one time, or not at all. For each bootstrap data set, the structure was retained but the coefficients and the intercept were reestimated. 

Note 505 bootstrap samples were generated for convenience. For standard nonparametric bootstrap, 200 replicates is adequate. 

For example, suppose jc = (1, 2, 3, 4, 5, 6, 7 with sample mean 4. The first bootstrap data set may consist of 1, 1, 4, 5, 3, 5, 3 with sample mean 3.14. The second bootstrap data set may consist of (1, 4, 1, 3,4, 2, 3 with sample mean 2.57. The bootstrap means, 3.14, 2.57, etc., can then be used to estimate the standard error of the observed sample mean and its corresponding Cl. Notice that not every number is represented in the bootstrap data set and that sometimes an observation is repeated these are properties of resampling with replacement. By resampling with replacement, an observation can be repeated or excluded. A key assumption of the bootstrap is that resampling (the simulation process) mimics the experimental procedure that gave rise to the observed data as closely as possible. In other words, as Fox (2002) has stated Thepopulation is to the sample as the sample is to the bootstrap sample. For example, if the data were correlated then resampling should mimic that correlation—it would not be valid to simulate assuming independence of the observations. 

So for a sample with 8, 9, and 10 observations there are 6,435, 24,310, and 92,378 possible combinations, respectively. With 2000 bootstrap resamples and 20 observations, the probability is greater than 0.95 that no bootstrap sample will repeat (Chernick, 1999). As a practical rule, bootstrapping would not be advised with less than 10 observations, although Chernick (1999) suggests that no fewer than 50 be used. With eight or fewer observations an exact bootstrap estimate can be developed using all possible bootstrap combinations (Fisher and Hall, 1991). 

Use one of many computer algorithms to randomly generate a set of values from this distribution (it should be the same number of observations as the original data set). This set of numbers is referred to as the bootstrap sample. 

Normal approximation method 9 1.96d, where 0 is the mean of the parameter of interest (actual data), B is the number of bootstrap samples, and d is the estimate of the standard error computed by... 

The nonparametric bootstrap is useful when distributions cannot be assumed as true or when the sampled statistic is based on few observations. In this setting, an observed data set, for example, Zj,. .., Z , where X could be vector-valued (i.e., concentrations at fixed sampling times) can be summarized in the usual way by a mean, median, and variance. An approximate sampling distribution can be obtained drawing a sample of the same size as the original sample from the original data with replacement, for example, Z/,. .., Z , where i is the index of the bootstrap sample... 

From the data we compute 0=X(6) = 12,900. To study the variation in this estimator we need to know its sampling distribution. We will use the bootstrap to approximate this distribution. We will generate first 200 bootstrap samples from F( ) and then 20,000 bootstrap samples using the following R code ... 

If we extended the simulation to 20,000 bootstrap samples, we obtain Average ... 

Thus, there were only minor differences in the mean and standard deviation for the sampling distribution of the median when comparing 200 bootstrap samples to 20,000 bootstrap samples. However, note the big discrepancies between the quantiles. When generating 20,000 samples of size 11 from the original dataset, samples were obtained in which the median of the bootstrap sample was equal to the minimum value (1600) in the original dataset. Because the bootstrap median equals 0 = X (6), this result implies that, in the bootstrap samples having median = 1600, at least 6 of the 11 data values must be equal to 1600. This seems very unlikely. However, if we calculate the expected number of samples in the 20,000 samples having exactly 6 of their 11 values equal to 1600, we find ... 

A plot of the quantile function, kernel density estimator of the p.d.f., a box plot, and a normal reference distribution plot for the sampling distribution of the sample quantile are given in Figures 2.13 and 2.14 for 200 and 20,000 bootstrap samples. We note that there are considerable differences in the plots. The plots for 20,000 bootstrap samples reveal the discreteness of the possible values for the median when the sample size (n = 11 in our case) is very small. Also, we note that n = 11 is too small for the sampling distribution for the median to achieve its asymptotic result (n large), an approximate normal distribution. 

The bagging algorithm uses bootstrap samples to build base classifiers. Each bootstrap sample is formed by randomly sampling, with replacement, the same number of observations as the training set. The final classification produced by the ensemble of these base classifiers is obtained using equal-weight voting. 

---