Pitfalls of poor uncertainty management

Because of improperly defined or missing uncertainties, undetected inconsistencies, neglected datasets, model oversimplification, and numerical problems it is not unusual for least-squares adjustment to force one or more of the model parameters outside its true uncertainty range without this being apparent to anyone. The error can easily be compounded in the literature if the incorrect parameter value is taken to be well-determined and used by others to help determine additional parameters. 

The solution of these problems lies in sharing models and data, allowing collaborative processing for validation and estimation with this information. As illustrated below, a lack of such collaborative makes it difficult, if not impossible, to combine raw, unprocessed information from distinct research groups, and consequently, the implications on distributed efforts to model complex systems are serious. 

In chemical kinetics modeling, we have seen artificial controversies arise between research groups solely due to the limited maimer in which research is reported and information is shared (e.g., concise, derived conclusions in archival journals). This has led to imprecise and inaccurate extraction of the information truly contained in the community s experimental data records. A typical situation in chemical kinetics goes as follows. In order to improve a complex model s predictive capability, scientist 1 devises an experiment Ei whose outcome should be dominantly 

Unfortunately, a completely symmetric train of thought simultaneously plays out in laboratory 2, performing a different experiment whose outcome is dominantly controlled by 02- Again, the minor dependence on 01, 03 will be ignored in the analysis.  

The circularity in the joint, uncoupled analysis is evident. Each party makes a simplifying assumption about precisely the quantity that the other party is studying in detail. For example, scientist 1 will use the literature reported value of 02 to update the literature reported value of 01, while scientist 2 does the opposite. The analysis is somewhat reminiscent of Gauss Seidel iteration, but the rate of convergence and the convergence itself to the correct model is not guaranteed. Something better should be used. 

---