Pre-processing

The raw images recorded by the camera have to be pre-processed to (1) correct for inevitable measurement deficiencies of the acquisition system, (2) reduce the amount of data that has to be processed to the necessary minimum and (3) prepare the data for the chosen evaluation algorithm. 

The second step is data reduction, by spectral range selection or selective binning of redundant data. The algorithms involved herein are optimised to reduce the data that has to be processed to the required minimum without losing relevant information. The proper execution of data reduction directly influences the real-time capabilities of the system. 

The final pre-processing step is data preparation for the evaluation. The particular operations are reliant on the chosen evaluation algorithm. Generally, a baseline 

A computational fluid dynamics process can be divided into three steps pre-processing, flow computation, and post-processing (Fig. 8.1). The users are faced with a multitude of commercial programs for each step. Newcomers in particular find it difficult to choose a suitable program, in part because the development of codes is advancing at a rapid pace. For many years, www.cfd-online.com has provided an excellent overview of CFD (suppliers, literature, further links). 

A hexahedral grid is the optimum solution. Commercial grid codes tend to fail in the complex intermeshing area between the two screws. 

---

D. Miller, D.Placko "Pre-processing of signals delivered by a new eddy current sensor for non ferromagnetic tubes testing" in IMTC 96... 

Inductive methods for establishing a correlation between chemical compounds and their properties are the theme of Chapter 9. In many cases, the structure of chemical compounds has to be pre-processed in order to make it amenable to inductive learning methods. This is usually achieved by means of structure descriptors, methods for the calculation of which are outlined in Chapter 8. 

To gain a general overview on data and its pre-processing for learning... 

First, we have an initial, and probably utterly crude, dataset. Genuine data pre-processing has only just started. The task is to assess the quality of the data. One of the topics for discussion in this chapter is the methods by which one finds out the potential drawbacks of the dataset. 

The two main ways of data pre-processing are mean-centering and scaling. Mean-centering is a procedure by which one computes the means for each column (variable), and then subtracts them from each element of the column. One can do the same with the rows (i.e., for each object). ScaUng is a a slightly more sophisticated procedure. Let us consider unit-variance scaling. First we calculate the standard deviation of each column, and then we divide each element of the column by the deviation. 

Mean-centering, as is shown by experience, can be successfully employed in combination with another data pre-processing technique, namely scaling, which is discussed later. 

If the task is multivariate calibration, for example, the proper choice of a pre-processing method will essentially aflFect the quality of the resultant model. For more details about the use of these techniques together with PCA and PLS, readers are advised to consider the fundamental monograph by Erikson et al [8]. 

It is necessary to pre-process data by mean-centering, scaling or autoscaling... 

The pre-processing concepts have been a more recent development of substructure searching systems. These approaches have become popular since the mid-1980s, when the cost of the storage devices (hard disks and CD-ROMs) decreased. 

Prior to PCA the data are often pre-processed to convert them into a form most suitable for the application of PCA. Commonly used pre-processing methods for PCA arc scaling and mcaii-ccntcring of the data, which arc described in Section 4.3. 

Raffenetti, R.C. Pre-processing two-electron integrals for efficient utilization in many-electron self-consistent field calculations. Chem. Phys. LeUera 20 335-338, 1973. 

Factor spaces are a mystery no more We now understand that eigenvectors simply provide us with an optimal way to reduce the dimensionality of our spectra without degrading them. We ve seen that, in the process, our data are unchanged except for the beneficial removal of some noise. Now, we are ready to use this technique on our realistic simulated data. PCA will serve as a pre-processing step prior to ILS. The combination of Principal Component Analysis with ILS is called Principal Component Regression, or PCR. 

Normalization, is an adjustment to a data set that equalizes the magnitude of each sample. In other words, normalization removes all information about the distance each data point lies from the origin of the data space but preserves the direction. Normalization has a relatively limited number of special applications. For example, it is frequently used a pre-processing step in preparing reference spectra for a qualitative identification library. The idea is to retain only the information that qualitatively distinguishes one sample from another while removing all information that could separate two samples of identical composition but different concentrations. 

Wilson J, Berry I. The use of gradient direction in pre-processing images from crystallization experiments. J Applied Cryst 2005 38 493-500. 

Another popular form of data pre-processing with near-infrared data is the application of the Multiplicative Scatter Correction (MSC, [28]). It is well known that particle size distribution of non-homogeneous powders has an overall effect on the spectrum, raising all intensities as the average particle size increases. Individual spectra x, are approximated by a general offset plus a multiple of a reference spectrum, z. 

The development of a calibration model is a time consuming process. Not only have the samples to be prepared and measured, but the modelling itself, including data pre-processing, outlier detection, estimation and validation, is not an automated procedure. Once the model is there, changes may occur in the instrumentation or other conditions (temperature, humidity) that require recalibration. Another situation is where a model has been set up for one instrument in a central location and one would like to distribute this model to other instruments within the organization without having to repeat the entire calibration process for all these individual instruments. One wonders whether it is possible to translate the model from one instrument (old or parent or master. A) to the others (new or children or slaves, B). 

The parameter values and patterns determined in this section are reasonably linked into adapted versions of data pre-evaluation procedures in order to permit automated pre-processing of complete series of scattering patterns. Novices at synchrotron beamlines will resort to on-site procedures and take home pre-processed data. 

The measured SAXS curve of the calibration sample must have been pre-processed in the usual way (cf. Sects. 7.3 - 7.6). Therefore it is important to have calibration samples with a well-defined thickness27. Because synchrotron beamlines can be adjusted to a fairly wide range of radiation power, it is important to have thin calibration samples for a high-power adjustment (e.g., common SAXS with wide slit openings) and thick calibration samples for low-power adjustments (e.g., USAXS with microbeam). For calibration samples from synthetic polymers, thicknesses ranging between 0.2 mm and 3 mm are reasonable. It appears worth to be noted that not only polymers, but as well glassy carbon [88] can be used as a solid secondary standard for the calibration to absolute intensity. 

Phytate (myo-inositol hexaphosphate Fig. 15.3, structure 33) is found in many food species and can be considered as a phytochemical. Its role in the plant is primarily as a phosphate store in seeds, but it is found in other tissues as well, for example, tubers (Harland et al., 2004). Phytate and its hydrolysis products are anti-nutrients that chelate metal ions and thus reduce their bioavailability (Persson et al., 1998 House, 1999). This is particularly a problem with cereal grains, but pre-processing can improve mineral absorption from these foods (Agte and Joshi, 1997). There is some concern that high phytate foods could also contain higher levels of toxic heavy metals caused by natural accumulation. Plants also contain phytate-degrading enzymes that can also influence metal ion bioavailability (Viveros et al., 2000). 

---