Reproducibility and Validation of Results

The major sources of poor reproducibility of results (i.e., Rp values, development time, zone shape, detection characteristics, etc.) in TLC can be broadly characterized as contamination, poor technique, and variations in chromatographic materials and/or conditions. Geiss (1987), Smith et al. (1973), and fork et al. (1990) listed the following parameters that can influence a TLC separation. These parameters should be recorded as completely as possible as part of the documentation procedure for TLC results (Chapter 9). 

The sorbent or stationary phase (type of sorbent brand batch number layer thickness particle size binder in layer precoated or homemade type of backing size of channels, if any activation method and temperature impregnation, if any) 

The solvent or mobile phase (the individual solvents, including any impurities pH method of preparing mobile-phase mixture volume and height of solvent pool in the chamber migration distance and time gradients, if any) 

Sample and standard solutions (weights solvent and volumes extraction and cleanup procedure for sample derivative preparation, if any location of samples and standards across the layer) 

Sample and standard application (spotting apparatus and technique, 

---

The validation was performed without any modifications. The procedure is very easy to reproduce and the achieved results correlate with the published material. The yield is somewhat lower than the published result, though monitoring of the reaction by TLC indicated complete consumption of substrate. This is believed to be due to decreased precipitation during recrystallization. Because the product is unstable in solution it is recommended that the recrystallization is performed as quickly as possible. Alternatively, the impurities can be removed by trituration. 

Despite the progress of exploratory research, the translation of pharmacogenomics from research to bedside has not been as rapid as we hoped. The major hurdle seems to be a lack of reproducibility and validation. The inconsistent results that currently exist in the literature are astonishing. These inconsistent results need to be confirmed by larger retrospective studies. Then, results from retrospective studies need to be verified in prospective studies. 

The characterization of the factors which control the accuracy, precision, and validity of measurements made in a simulation facility for studying in-cloud chemical processes was described. An analysis of a large number of experimental data collected under widely varying conditions was performed. Cloud liquid water content, an observable principally dependent on cooling rate and reaction time, was found to be the most influential of the physical factors controlling the resultant chemistry. In order to precisely control and reproduce the physical conditions in the simulation facility, standard operating procedures and computer control were instituted. This method reduced the uncertainty of the SO2 to sulfate transformation rate by a factor of 4.4. 

The kinetic equations serve as a bridge between the microscopic domain and the behavior of macroscopic irreversible processes through the description of hydrodynamics in terms of intermolecular collisions. Hydrodynamics can specify a large number of nonequilibrium states by a small number of reproducible properties such as the mass, density, velocity, and energy density of a fluid conserved during the collision of molecules. Therefore, the hydrodynamic equations can describe a wide range of relaxation processes of nonequilibrium states to equilibrium state. We call such processes decay processes represented by phenomenological equations, such as Fourier s law of heat conduction. The decay rates are determined by the transport coefficients. Reliable transport coefficients provide microscopic and macroscopic information, and validate the results of molecular dynamics. 

This will be subject to extensive testing as more data accumulates, other workers try to reproduce and extend the results, and ancillary information is brought to bear on the issue. During this process it is important not to let unverified or scant contrary results dominate discussion on the validity of a hypothesis. Contrary results can be indicative of several problems. For example ... 

The recent literature on microwave-assisted chemistry has reported a multitude of different effects in chemical reactions and processes and attributed them to microwave radiation. Some of these published results cannot be reproduced, however, because the household microwave ovens employed often have serious technical shortcomings. Published experimental procedures are often insufficient and do not enable reproduction of the results obtained. Important factors required for qualification and validation, for example exact records, reproducibility, and transparency of reactions/processes, are commonly not reported, which poses a serious drawback in the industrial development of microwave-assisted reactions and processes for synthesis of fine chemicals, intermediates, and pharmaceuticals. Technical microwave devices for synthetic chemistry have been on the market for a while (cf a.m. explanations) and should enable comparative investigations to be conducted under set conditions. These investigations would enable better assessment of the observed effects. It is, furthermore, possible to obtain a better insight into the often discussed (nonthermal) microwave effects from these experiments (Ref. [138] and Chapter 4 of this book). Technical microwave systems are an important first step toward the use of microwave energy for technical synthesis. The actual scale-up of chemical reactions in the microwave is, however, still to be undertaken. Comparisons between microwave systems with different technical specifications should provide a measure for qualification of the systems employed, which in turn is important for validation of reactions and processes performed in such commercial systems. 

Representations of precision (reproducibility) and validity (accuracy). The target for accuracy of a test result is represented by the central disc. For example, this is the correct mean for a sample analyzed by ELISA. Results from five tests are shown as black dots. (A) Data are grouped tightly (reproducible) and all are in the correct result disc (accurate). (B) Although the data are precise (all showing similar results), they all are inaccurate and biased toward the upper left. (C) This shows a wide dispersion of data (not reproducible), but the average result of the data predicts the accurate result confidence in these data is low. (D) This shows both irreproducible results in which the control mean is not reflected by the average of all the results the test is inaccurate... 

The most recent bioanalytical Workshop Report (Viswanathan 2007) devotes considerable space to this topic and some recommendations not discussed previously (Section 9.4.7b) are included below. There should be some assessment of both reproducibility and accuracy of the reported concentration. Sufficient data should be generated to demonstrate that the current matrix (i.e. the incurred sample matrix) produces results similar to those previously validated. It is recognized that accuracy of the result generated from incurred samples can be more difficult to assess. It requires evaluation of any additional factors besides reproducibility upon storage, which could perturb the reported concentration. These could include metabolites converted to parent during sample preparation or LC-MS/MS analysis, matrix effects from high concentrations of metabolites, or variable recovery between analyte and internal standard (Viswanathan 2007). Most of these phenomena are those described previously (Jemal 2002) and discussed in Section 9.4.7b. 

We studied how the dimensions of test specimens affect EWF results. One grade of commercial HDPE was evaluated. We demonstrate that reductions of 20 to 30% of conventional dimensions used for DDENT geometry allow reproducible and valid results, similar to those obtained with larger specimens. Results show that it is not possible to reduce dimensions more than 30% and still comply with geometric validations related to EWF. 

In the nonclassical ion controversy discussed in Chapter 9, there was never any question on either side of the debate about the validity of the observed data, only about their interpretation. Had any of the experimental data been questioned or found to be incorrect, this would have been soon found out because so many people repeated and rechecked the data. This is the strength of science (in contrast to politics, economics, etc.), i.e., that we deal with reproducible experimental observation and data. Nevertheless, interpretation can still result in heated discussions or controversies, but science eventually will sort these out based on new results and data. 

Validation of a force field is typically done by showing how accurately it reproduces reference data, which may or may not have been used in the actual parameterization. Since different force fields employ different sets of reference data, it is difficult to compare their accuracy directly. Indeed there is no single best force field, each has its advantages and disadvantages. They perform best for the type of compounds used in the parameterization, but may give questionable results for other systems. Table 2.6 gives some typical accuracies for AH( that can be obtained with the MM2 force field. 

At sufficiently low strain, most polymer materials exhibit a linear viscoelastic response and, once the appropriate strain amplitude has been determined through a preliminary strain sweep test, valid frequency sweep tests can be performed. Filled mbber compounds however hardly exhibit a linear viscoelastic response when submitted to harmonic strains and the current practice consists in testing such materials at the lowest permitted strain for satisfactory reproducibility an approach that obviously provides apparent material properties, at best. From a fundamental point of view, for instance in terms of material sciences, such measurements have a limited meaning because theoretical relationships that relate material structure to properties have so far been established only in the linear viscoelastic domain. Nevertheless, experience proves that apparent test results can be well reproducible and related to a number of other viscoelastic effects, including certain processing phenomena. 

From Table 2 it is observed that the dispersive NIR ensembles (NIR and NIR R) result in the best cross validated models. The potential advantages of Fourier transform spectroscopy [5] are in practice outnumbered by a more reproducible setup and saimpling procedures. 

---