Unsolved Problems in Chemometrics

Conspicuous by its absence is the question of calibration transfer, even though we consider it unsolved in the general sense, in that there is no single recipe or algorithm that is pretty much guaranteed to work in all (or at least a majority) of cases. Nevertheless, not only are many people working on the problem (so that it is hardly unaddressed ), but there have been many specific solutions developed over the years, albeit for particular calibration models on particular instruments. So we do not need to beat up on this one by ourselves. 

1) The first one we mention is the question of the validity of a test set. We all know and agree (at least, we hope that we all do) that the best way to test a calibration model, whether it is a quantitative or a qualitative model, is to have some samples in reserve, that are not included among the ones on which the calibration calculations are based, and use those samples as validation samples (sometimes called test samples or prediction samples or known samples). The question is, how can we define a proper validation set Alternatively, what criteria can we use to ascertain whether a given set of samples constitutes an adequate set for testing the calibration model at hand  

A very limited version of this question, does in fact, sometimes appear, when the question arises of how many samples from a given calibration set to keep in reserve for 

Even so, at best any of these answers treat only one aspect of the larger question, which includes not only how many samples, but which ones A properly taken random sample is indeed representative of the population from which it comes. So one subquestion here is, how should we properly sample The answer is randomly but how many workers select their validation samples in a verifiably random manner How can someone then tell if their test set is then valid, and against what criteria  

Some of this goes back to the original question of obtaining a proper and valid set of calibration samples in the first place, but that is a different, although related problem. We can turn that question around in the same way what are the criteria for telling if a calibration sample set is a valid set Maybe both problems have the same solution, but we do not know because nobody is working on either one. 

---

There are innumerable unsolved problems in Chemometrics that need to be addressed real, scientific problems, not just new ways to throw numbers around. 

---