Benchmarking Data Set

a larger set of 318 non-homologous chains representative of high-resolution structures became available (Sali Overington, 1994). All X-ray determined structures had a resolution of 2.3A or better (although some NMR-determined structures were also used), and no protein chains contained more than 30% sequence identity. All residues assigned by the DSSP program that were neither a helix (H) nor extended P sheet (E) 

The commonly used benchmark for protein fold recognition is the test set suggested by Fischer et al. (1996), which comprises a set of 68 pairs of proteins with very low sequence similarity, but very highly similar folds. 

---

Use of this damping term within the D3 correction -D3(BJ) is the method now promoted by Grimme. Mean deviations for a benchmark data set that involve weak interactions, chemical reactions, and conformations are reduced by at least 1 kcal mol with the inclusion of the D3 corrections for a range of functionals. Even the performance of the double hybrid methods can be improved by about... 

A 20-variable process data (temperature recordings only) is considered. This benchmark data set is composed of measurements recorded from temperature sensors. The first 450 measurements are used to build the PCA model and the remaining 342 samples are utilized as a test set. Known disturbances reported for the test set are given in Table 8.2. It was pointed out that not all the disturbances in the test set have been previously identified. 

Numerical examples (more than 150) and several tables listing molecular descriptors for two benchmark data sets are added to help students and nonexpert readers to comprehend the algorithms better. Indeed, this new edition has been conceived not... 

There exists a simple 2x2x2 array which shows degeneracy problems and often serves as a benchmark data set to test algorithms [Kruskal et al. 1983, ten Berge Kiers 1988], This array X has frontal slabs Xi and X2 ... 

Coats EA. The CoMFA steroids as a benchmark data set for development of 3D-QSAR methods. In Kubinyi H, Folkers G, Martin YC, eds. 3D QSAR in Drug Design. Vol. 3. Recent Advances. Dordrecht Kluwer/ESCOM, 1998 199-213. 

This section presents an in-depth discussion on the forecasting performance of the proposed HI model. The forecasting performance of the proposed HI model is first analyzed based on the experimental results presented in Section 9.4. Further analysis is then conducted to validate the superiority of the proposed model over other models based on public benchmark data sets. The effectiveness of the model s components, including the heuristic fine-tuning process, data preprocessing component and HI forecaster, is also analyzed in this section. 

Table 9.12 Comparison of annual forecasting results (benchmark data sets)...

Evaluation of systems A common problem in the activity-recognition community is the lack of annotated reference data and standardized test beds that could help researchers compare the performances of their approaches. Although there exist several benchmark data sets, most of these data sets consist of simple activities, such as walking, running, sitting, and sleeping. Very few common data sets exist... 

In general, historic experiments were performed primarily to explore characteristics and behaviors of tsunamis. On the other hand, a majority of recent experiments aims at providing adequate benchmark data sets for validation of muneric models. For example, benchmarking exercises for numeric models were conducted at the 1995 Friday Harbor Workshop and the 2004 Catalina Island Workshop Objectives of laboratory experiments have evolved together with advances in measuring instruments. 

J.M.L. (2008) Highly accurate first-principles benchmark data sets for the parametrization and validation of density functional and other approximate methods. Derivation of a robust, generally applicable, double-hybrid functional for thermochemistry and thermochemical kinetics.. Phys. Chem. A, 112, 12868-12886. 

During the last years, more and more researchers have applied density functional theory to small transition-metal complexes and benchmarked the results against either high level wave function based methods or experimental data. A particular set of systems for which reasonably accurate benchmark data are available are the cationic M+-X complexes, where X is H, CH3 or CH2. Let us start our discussion with the cationic hydrides of the 3d transition-metals. 

The recent versions of the slow motion approach were applied to direct fitting of experimental data for a series of Ni(II) complexes of varying symmetry (97). An example of an experimental data set and a fitted curve is shown in Fig. 9. Another application of the slow-motion approach is to provide benchmark calculations against which more approximate theoretical tools can be tested. As an example of work of this kind, we wish to mention the paper by Kowalewski et al. (98), studying the electron spin relaxation effects in the vicinity and beyond the Redfield limit. 

The above findings relate to one assay and one data set built up over time, possibly with many analogs of active compounds, and the findings might be different for commercial molecular databases. More benchmarking along these lines is needed. 

In the next two subsections, we describe collections of calculations that have been used to probe the physical accuracy of plane-wave DFT calculations. An important feature of plane-wave calculations is that they can be applied to bulk materials and other situations where the localized basis set approaches of molecular quantum chemistry are computationally impractical. To develop benchmarks for the performance of plane-wave methods for these properties, they must be compared with accurate experimental data. One of the reasons that benchmarking efforts for molecular quantum chemistry have been so successful is that very large collections of high-precision experimental data are available for small molecules. Data sets of similar size are not always available for the properties of interest in plane-wave DFT calculations, and this has limited the number of studies that have been performed with the aim of comparing predictions from plane-wave DFT with quantitative experimental information from a large number of materials. There are, of course, many hundreds of comparisons that have been made with individual experimental measurements. If you follow our advice and become familiar with the state-of-the-art literature in your particular area of interest, you will find examples of this kind. Below, we collect a number of examples where efforts have been made to compare the accuracy of plane-wave DFT calculations against systematic collections of experimental data. 

A benchmark dose can be defined from a data set that does not include a NOAEL. 

For the derivation of EQSs (and similar benchmarks), experimental toxicity data are considered essential. However, for many substances there will be insufficient reliable toxicity data available to meet the prescribed minimum data requirements. In their absence (or to supplement an existing data set), several extrapolative methods may potentially be of assistance. Nevertheless, we recommend extreme caution when extrapolating from calculated values to predicted real toxicity data. Most suggested calculation methods to supplement missing toxicological data are considered unacceptable in EQS derivation. 

The benchmark calculations of ionization potentials and electron affinities of the atoms and molecules in the G2 data set" calculated using the hybrid functional (B97) show that this functional is adequate. The average absolute deviation from experimental data amounts to 0.055 eV and 0.056 eV for ionization potential and electron affinity, respectively.51... 

It should be emphasized that these methods will generally be considered for an acute lethal endpoint. Their use to set AEGL-1 and AEGL-2 values will be considered on a chemical-by-chemical basis. Different endpoints may require the use of different data sets in different or the same species, a different benchmark dose approach, or identification of a different response level. These factors wiU be considered for specific chemicals and toxicologic end-points. 

---