As the need of crossing multiple disciplines in current research, a great number of scientists are trying to develop some complex mathematic model for combining the knowledge from different fields, climate change impact assessment study being the one of good examples. In such research, people typically couple one crop model (just an example, you can change any other model depending on your research interests) with climate model, which are referred as impact model (very complex, of course), to make an assessment under further climate scenario by adjusting the relevant model parameters. Here this approach can be called Top-Bottom approach, which means they start from model.
However, recently, some studies based on Bottom-Up approach (different the above notion, it starts from observations) showed a different result and casted serious doubt on the predictability of these complex mathematic model. For example, Spencer and Braswell (2008), through a simple model, demonstrated a potential bias in temperature simulated by climate model because early investigator ignored an interaction between cloudiness and temperature in traditional climate model. I also published a paper in Agricultural and Forest Meteorology recently (Zhang et al., 2008) in which I concluded that the strong negative impact of temperature to lowland rice yield is overestimated because of an implicit assumption in crop model. Even though the two cases are in different fields the scientific meaning behind them is actually similar; there are so many underlying assumptions in model, the more complex model is, the more such assumptions is there. But mistake of only one assumption can lead to a miscalculation of model, then resulting in a bad prediction. Admittedly, these studies seem controversial now but they do reflect the inherent weakness of complex mathematic models.
Yesterday a professor in the Netherlands interviewed me for a position in his research group. His research is to develop a model that includes all knowledge from bio physical and social disciplines. I just asked this question and he fully agreed with me. It is difficult to evaluate a model if the model is trying to work on bio physical and socioeconomic disciplines. We will never know how the model performances because we are not able to figure out an efficient method to separate the individual impact from an observed trend mixing with so many impacts. Then how can we believe the model output without a strict evaluation? Of course I know model has to be evaluated by comparing simulated value and observed value before model scenario simulation. This evaluation method is too simple to evaluate a model for ensuring a good prediction even though everybody used that. Please read the two articles I listed in the above paragraph you will understand.
This made me think some questions. Do we look at model output in an appropriate way? Will the complex model we are using or developing help us or mislead us?
I am looking forward your creative thinking over the topic. Thank you
PS: Sorry for any confusion as wrong spell or bad expression. English is not my native language.
Spencer, R. W., Braswell, W.D. 2008. Potential biases in feedback diagnosis from observational data: A simple model demonstration. Journal of climate, 21, 5624 – 5628
Zhang, T., Zhu, J., Yang, X. 2008a. Non-stationary thermal time accumulation reduces the predictability of climate change effects on agriculture, Agric. For. Meteorol. 148, 1412–1418