Estimation and detection of flexible trends
Atmospheric Environment, 38 (25):
4135--4145 (2004)DOI: 10.1016/j.atmosenv.2004.04.014
Abstract
Literature on the estimation and detection of linear trends in (environmental) time series is extensive, as overviewed by Hess and others in work published in 2001. However, not all data patterns behave in a linear fashion or, at least, a monotonous increasing or decreasing fashion. Therefore, it may be advantageous to analyse data with more flexible trends. Once flexibility is introduced the problem of detecting statistically significant increase/decrease becomes much more complicated: the trend may be alternating between being constant, decreasing or increasing. As a consequence, the problem of detecting significant increases or decreases cannot be summarised in one single figure or statistical test, as in the case of linear trend detection.
Here, three functions of time are proposed so as to grasp the uncertainty in flexible trend estimates, denoted as μt, with t=1,…,N. These functions are: (i) the actual trend estimate μt, plus its uncertainty, (ii) the first difference in the trend μt–μt−1, with its uncertainty and (iii) the lagged difference μN–μt, with its uncertainty. These three functions can be argued as giving enough practical information on significance for most environmental applications. Furthermore, it is shown that these confidence limits can by estimated within the Kalman-filter framework of structural time-series models. The reason for choosing trends from this class of models is not that they are necessarily ‘better’ than other trend models but the sole presence of mathematical formulae for the uncertainty in the differences μt–μs with 1⩽s⩽t⩽N. The choice of a proper trend model is not trivial and is addressed in an appendix. The approach is illustrated using the software package TrendSpotter, which is available from the author.