Data Filtering and Outliers

The filtering of 1D signals is commonly achieved using fft, spline functions, wavelets etc. [61]. The goal of filtering is to increase signal to noise. However, in spectroscopy, loss of resolution and spectral distortion can arise due to over-filtering. 

Each batch experiment or each step of a semi-batch experiment results in a data array Ai, . This matrix is often rather smooth in the time direction, especially if rather fast spectra are taken compared to the evolving chemistry. If one follows any particular channel of data in the time direction, it may slowly increase or de- 

1) All our group s algorithms have been implemented in MatLab on PCs or workstations. MatLab is a very convenient platform for importing, manipulating and exporting spectroscopic data. The MatLab Ubrary of functions is 

The 2D property can be used to increase filtering efficiently [62]. We have filtered FTIR data from the homogeneous catalyzed rhodium hydroformylation of alkenes using a variety of ID and 2D filters. On blocks of 100-1000 spectra, the ID filters i. e. SG, fft, cubic spline, can reduce noise by ca. 10-50%, but the 2D filters, i. e. 2D fft, can reduce the noise level even further, to ca. 85 %+ [63]. The procedure for each block of spectroscopic data can be viewed as Eq. (7) 

---