The Jackknife

For simplicity 9 will be assumed to be one dimensional, but what will be discussed below applies to multivariate 9 as well. First, define 9 and 9, as the statistic of interest with and without the ith observation, respectively. Then, define the ith pseudo value for 9 as 

The square root of is the jackknife standard error of the estimate. This method is called the delete-1 jackknife because a single observation is removed at a time. A modification of this method, called the delete-n jackknife, is to delete chunks of data at a time and then create the pseudovalues after removal of these chunks. 

The jackknife has a number of advantages. First, the jackknife is a nonparametric approach to parameter inference that does not rely on asymptotic methods to be accurate. A major disadvantage is that a batch or script file will need to be needed to delete the ith observation, recompute the test statistic, compute the pseudovalues, and then calculate the jackknife statistics of course, this disadvantage applies to all other computer intensive methods as well, so it might not be a disadvantage after all. Also, if 9 is a nonsmooth parameter, where the sampling distribution may be discontinuous, e.g., the median, the jackknife estimate of the variance may be quite poor (Pigeot, 2001). For example, data were simulated from a normal distribution with mean 100 and 

Consider the univariate case where a random variable X is measured n-times and some statistic f(x) is calculated from the sample vector X. In its most basic form, the nonparametric bootstrap is done as follows  

2 Coverage is the proportion or percent of times a Cl contains the true parameter of interest. A 95% Cl should contain the true parameter 95% of the time—this is its nominal coverage. 

---

Efron B (1982) The jackknife, the bootstrap and other resampling techniques. Society for Industrial and Applied Mathematics, Philadelphia, PA... 

Two non-parametric methods for hypothesis testing with PCA and PLS are cross-validation and the jackknife estimate of variance. Both methods are described in some detail in the sections describing the PCA and PLS algorithms. Cross-validation is used to assess the predictive property of a PCA or a PLS model. The distribution function of the cross-validation test-statistic cvd-sd under the null-hypothesis is not well known. However, for PLS, the distribution of cvd-sd has been empirically determined by computer simulation technique [24] for some particular types of experimental designs. In particular, the discriminant analysis (or ANOVA-like) PLS analysis has been investigated in some detail as well as the situation with Y one-dimensional. This simulation study is referred to for detailed information. However, some tables of the critical values of cvd-sd at the 5 % level are given in Appendix C. 

The jackknife estimate of variance can be used to assess the significance of the weight and loading coefficients in a PLS model. This is a valuable source of information in case interpretation of the parameters is warranted. The weights with jackknifed standard deviations from the BHT example are given in Table 6.5. The weights of the first PLS dimension suggest that the behavioural effect of these doses of BHT is to suppress most aspects of a rat s... 

A second possibility is to use some estimate of the variance of the loadings. This can be done by the jackknife method due to Quenouille and Tukey (see [37]) or by Efron s bootstrap method [38] (the colourful terminology stems from the expressions jack of all trades and master of none and lifting yourself up by your own bootstraps ). The use of the bootstrap to estimate the variance of the loadings in PCA has been described [39] and will not be elaborated upon further. The jackknife method is used partly because it is a natural side-product of the cross-validation and therefore computationally non-demanding and partly because the jackknife estimate of variance is used later on in conjunction with PLS. 

The jackknife method is based on an idea similar to cross-validation. The calculation of the statistical model is repeated g times holding out 1/gth of the data each time. In the end, each element has been held out once and once only (exactly as in cross-validation). Thus, a number of estimates of each parameter is obtained, one for each calculation round. It has been proposed that the quantity... 

Efron B (1981) Nonparametric estimates of standard error The jackknife, the bootstrap and other methods. Biometrika 68 589-599... 

Jackknife estimate of the bias. For the estimate discussed above, consider the jackknifed samples obtained by deleting one point from an original sample, — Let F k) be the estimated free energy obtained with the Mi jackknifed sample so that... 

Fig. 6. Repacking of the influenza HA2 hydrophobic core. Left. A ribbon trace of HA2 residues 38 to 127, including the helices that make up the core of the stalk in the native HA structure (see Fig. 3). Middle A hypothetical structure obtained by fusing the base of the coiled coil from the native HA structure with the top of the extended coiled coil from the low pH-converted HA structure. This panel helps distinguish the two major components of the HA conformational change on low pH treatment the existence of such an intermediate structure has not been shovm experimentally for influenza and may exist only transiently if at all. This extended structure, known as a prehairpin intermediate, has been detected indirectly in other virus envelope proteins (reviewed in Chan and Kim, 1998). Right Residues 38 to 127 from low pH-converted HA2 (Bullough et al, 1994). Hydrophobic residues that stabilize the jackknifed structure are indicated in one protomer as gray space-filling atoms. The amino (N) and carboxy (C) termini of a protomer within each trimer structure are indicated.

Figure 14-30 An example of application of weighted Deming regression analysis.The solid line is the estimated regression line, and the dotted line is the line of identity.The estimated 95% confidence bands obtained by the jackknife approach are the curved dashed lines.

The slope and intercept may be estimated by a nonpara-metric procedure, which is robust to outliers, and requires no assumptions of Gaussian error distributions. Notice, however, that the parametric regression procedures do not presume Gaussian distributions of target values, but only of the error distributions. Furthermore, the jackknife principle used for estimation of standard errors for Deming and... 

SE(flo) and SE b) are the standard errors of the estimated intercept Oo and slope b, respectively. For OLR and WLR, the standard errors are calculated from the formulas presented previously. These formulas also apply approximately for the Deming and weighted Deming procedures. An exact procedure is to apply a computerized resampling principle called the jackknife procedure, which in practice can be carried out... 

When a model is used for descriptive purposes, goodness-of-ht, reliability, and stability, the components of model evaluation must be assessed. Model evaluation should be done in a manner consistent with the intended application of the PM model. The reliability of the analysis results can be checked by carefully examining diagnostic plots, key parameter estimates, standard errors, case deletion diagnostics (7-9), and/or sensitivity analysis as may seem appropriate. Conhdence intervals (standard errors) for parameters may be checked using nonparametric techniques, such as the jackknife and bootstrapping, or the prohle likelihood method. Model stability to determine whether the covariates in the PM model are those that should be tested for inclusion in the model can be checked using the bootstrap (9). 

Furthermore, when alternative approaches are applied in computing parameter estimates, the question to be addressed here is Do these other approaches yield similar parameter and random effects estimates and conclusions An example of addressing this second point would be estimating the parameters of a population pharmacokinetic (PPK) model by the standard maximum likelihood approach and then confirming the estimates by either constructing the profile likelihood plot (i.e., mapping the objective function), using the bootstrap (4, 9) to estimate 95% confidence intervals, or the jackknife method (7, 26, 27) and bootstrap to estimate standard errors of the estimate (4, 9). When the relative standard errors are small and alternative approaches produce similar results, then we conclude the model is reliable. 

Very often a test population of data is not available or would be prohibitively expensive to obtain. When a test population of data is not possible to obtain, internal validation must be considered. The methods of internal PM model validation include data splitting, resampling techniques (cross-validation and bootstrapping) (9,26-30), and the posterior predictive check (PPC) (31-33). Of note, the jackknife is not considered a model validation technique. The jackknife technique may only be used to correct for bias in parameter estimates, and for the computation of the uncertainty associated with parameter estimation. Cross-validation, bootstrapping, and the posterior predictive check are addressed in detail in Chapter 15. 

B. Efron, Bootstrap methods another look at the jackknife. Ann Stat 7 1-26 (1979). 

B. Efron and G. Gong, A leisurely look at the bootstrap, the jackknife and cross-validation. Am Statistician 37 36-48 (1983). 

The reliability of the parameter estimates can be checked using a nonparametric technique—the jackknife technique (20, 34). The nonlinearity of the statistical model and ill-conditioning of a given problem can produce numerical difficulties and force the estimation algorithm into a false minimum. 

The preciseness of the primary parameters can be estimated from the final fit of the multiexponential function to the data, but they are of doubtful validity if the model is severely nonlinear (35). The preciseness of the secondary parameters (in this case variability) are likely to be even less reliable. Consequently, the results of statistical tests carried out with preciseness estimated from the hnal ht could easily be misleading—thus the need to assess the reliability of model estimates. A possible way of reducing bias in parameter estimates and of calculating realistic variances for them is to subject the data to the jackknife technique (36, 37). The technique requires little by way of assumption or analysis. A naive Student t approximation for the standardized jackknife estimator (34) or the bootstrap (31,38,39) (see Chapter 15 of this text) can be used. 

M. H. Quenouille introduced the jackknife (JKK) in 1949 (12) and it was later popularized by Tukey in 1958, who first used the term (13). Quenouille s motivation was to construct an estimator of bias that would have broad applicability. The JKK has been applied to bias correction, the estimation of variance, and standard error of variables (4,12-16). Thus, for pharmacometrics it has the potential for improving models and has been applied in the assessment of PMM reliability (17). The JKK may not be employed as a method for model validation. 

Efron B, Gong G, A leisurely look at the Bootstrap, the Jackknife, and the cross-validation, The American Statistician, 1983, 37, 36-48. 

Efron B, The Jackknife, the Bootstrap and Other Resampling Plans, Society for industrial and applied mathematics, Philadelphia, PA, 1982. 

Related to the bootstrap is the jackknife approach (see the book Appendix for further details and background) of which there are two major variants. The first approach, called the delete-1 approach, removes one subject at a time from the data set to create n-new jackknife data sets. The model is fit to each data set and the parameter pseudovalues are calculated as... 

For large data sets, the delete-1 jackknife may be impractical since it may require fitting hundreds of data sets. A modification of the delete-1 jackknife is the delete 10% jackknife, where 10 different jackknife data sets are created with each data set having a unique 10% of the data removed. Only 10 data sets are modeled using this jackknife modification. All other calculations are as before but n now becomes the number of data sets, not the number of subjects. The use of the jackknife has largely been supplanted by the bootstrap since the jackknife has been criticized as producing standard errors that have poor statistical behavior when the estimator is nonsmooth, e.g., the median, which may not be a valid criticism for pharmacokinetic parameters. But whether one is better than the other at estimating standard errors of continuous functions is debatable and a matter of preference. 

---