Principle of Parsimony

The principle of parsimonious parametrization should be applied, according to which the model with as few parameters as possible should be selected this is in accordance with the Wheeler recommendation the best model is the simplest one that works . This is also a guideline for making increasingly complex models stop when the model is sufficiently complex to get an adequate fit of the data. 

By slowly increasing the complexity of the models in this fashion, it was hoped that a model could be obtained that was just sufficiently complex to allow an adequate fit of the data. This conscious attempt to select a model that satisfies the criteria of adequate data representation and of minimum number of parameters has been called the principle of parsimonious parameterization. It can be seen from the table that the residual mean squares progressively decrease until entry 4. Then, in spite of the increased model complexity and increased number of parameters, a better fit of the data is not obtained. If the reaction order for the naphthalene decomposition is estimated, as in entry 5, the estimate is not incompatible with the unity order of entry 4. If an additional step is added as in entry 6, no improvement of fit is obtained. Furthermore, the estimated parameter for that step is negative and poorly defined. Entry 7 shows yet another model that is compatible with the data. If further discrimination between these two remaining rival models is desired, additional experiments must be conducted, for example, by using the model discrimination designs discussed later. The critical experiments necessary for this discrimination are by no means obvious (see Section VII). 

Table XII would indicate that the effect of water is not described adequately by the model. Utilizing the principle of parsimonious parameterization, one can consider both water and carbon dioxide to be adsorbed and oxygen to be nonadsorbed, resulting in the three-parameter model 4. The residuals in Table XII for model 4, however, are correlated with the oxygen level. Hence model 5 would perhaps be preferable, for it likewise contains only three parameters while allowing adsorbed oxygen. The random residuals of Table XII for model 5 indicate that this model cannot be rejected using the...

The belief is that the statistical method used (such as PLS, PCR, MLR, PCA, ANNs) will extract from the data those variables which are most important, and discard irrelevant information. Statistical theory shows that this is incorrect. In particular, the principle of parsimony states that a simple model (one with fewer variables or parameters), if it is just as good at predicting a particular set of data as a more complex model, will tend to be better at predicting a new, previously unseen data set [153-155]. Our work has shown that this principle holds. 

The choice of the right model to use to describe experimental results is one of the trickiest, and most interesting, tasks in scientific work, and this is a subject that can only be touched on here. As discussed above, we are guided by the Principle of Parsimony, that in science one should seek the simplest explanation for phenomena. In the present context, that means that we should define models with as few parameters as possible, consistent with obtaining a satisfactory description of the data. This is a sensible approach, because if a simple model fits the data adequately, then so necessarily must more complicated versions of that model. It follows that experimental observations can only serve to rule out models, often, but not always, because they are oversimplified the data can never prove that a model is correct The question naturally arises at this stage about how one can establish whether or not a model is successful in accounting for the data. There are several criteria for assessing the quality of a model. 

Some of the first ideas on multi-way analysis were published by Raymond Cattell [1944,1952], Thurstone s principle of parsimony states that a simple structure should be found to describe a data matrix or its correlation matrix with the help of factors [Thurstone 1935], For the simultaneous analysis of several matrices together, Cattell proposed to use the principle of parallel proportional profiles [Cattell 1944], The principle of parallel proportional profiles states that a set of common factors should be found that can be fitted with different dimension weights to many data matrices at the same time. This is the same as finding a common set of factors for a stack of matrices, a three-way array. To quote Cattell ... 

The principle of parsimony, it seems should not demand Which is the simplest set of factors reproducing this particular correlation matrix but rather Which set of factors will be most parsimonious at once to this and other matrices considered together ... 

The next two properties, appropriate level of detail and as simple as possible, are two sides of the same coin because model detail increases at the expense of simplicity. Modeler s refer to this aspect of model development as Occam s razor. Formulated by William of Occam in the late Middle ages in response to increasingly complex theories being developed without an increase in predictability, Occam s razor is considered today to be one of the fundamental philosophies of modeling—the so-called principle of parsimony (Domingos, 1999). As ori-... 

Principle of parsimony (Occam s Razor William of Ockham, 1285-1349/ 1350, English philosopher and logician). All things being (approximately) equal, one should accept the simplest model. 

The combinatorial approach that was pursued in search of an antiasthma drug based on a split-and-mix strategy [92] as a practical use of the operational principle of parsimony was to get the most with the least in this case, to get 343 different types of variants in only 21 reaction steps. Scheme 1-17 sketches... 

Thus, as expected, the design, operation, and performance of the EKR system are not easy. Mathematical models are necessary in order to gain a better understanding of the processes that occurs in the EKR and to allow predictions for the field-scale remediation. Generally, it is a good policy to keep the mathematical model as simple as possible while adequately describing the behavior of the main parameters of the system (principle of parsimony). Thus, models with relatively simple transport equations and few equilibrium equations are able to predict the evolution of parameters such as the rate of recovery of the toxic ion, the maximum recovery, the rate of acid addition, and the energy requirements. The equation of mass conservation for a pore water solute species (e.g. an ion) in an EKR system can be expressed as follows ... 

Eqs. 63 — 68 reveal a typical dilemma in Hansch analysis while eqs. 65 — 68 are significantly better than eq. 63 and are based on more reasonable assumptions than eq. 64, which one of them is the best equation On the basis of the correlation coefficients r (the crutches of a QSAR beginner), eq. 67 is to be preferred eq. 66 seems to be the best one if the standard deviation s, a much better criterion, is considered. The differences in the correlation coefficients r and in the standard deviations s of eqs. 65—68 are rather small. However, if one applies the principle of parsimony, eqs. 66 and 67 should be omitted because too many parameters are included for such a small data set. 

The search for the shortest trees in tree space and the use of the cladistic version of the principle of parsimony, as implemented in computer programs... 

Algorithms that implement a cladistic application of the principle of parsimony allowing the use of exact optimality criteria and (in principle) the exact search for the shortest trees... 

The conclusion from this reasoning is that cladistic application of the principle of parsimony implies an important first principle, namely that for equally weighted characters the probability of homology for each putative novelty should be the same for all characters. Only this justifies counting each change in the same way. The innate existence of this assumption in cladistics, and its methodical consequences, has been ignored in the past by many cladists. 

The principle of parsimony degenerates to numerical taxonomy, if characters are not weighted. (Remane, 1983)... 

The principle of parsimony (de Noord, 1994 Flury and Riedwyl, 1988 Seasholtz and Kowalski, 1993) states that if a simple model (that is, one with relatively few parameters or variables) fits the data then it should be preferred to a model that involves redundant parameters. A parsimonious model is likely to be better at prediction of new data and to be more robust against the effects of noise (de Noord, 1994). Despite this, the use of variable selection is still rare in chromatography and spectroscopy (Brereton and Elbergali, 1994). Note that the terms variable selection and variable reduction are used by different researchers to mean essentially the same thing. 

---