Critical phenomena in portfolio selection
Abstract Book of the XXIII IUPAP International Conference on Statistical Physics, Genova, Italy, (9-13 July 2007)Abstract
Rational portfolio selection is seeking a tradeoff between risk and expected return, by optimizing the risk functional over the portfolio weights, given the expected return, the budget constraint, and possibly other constraints. In real life the risk functional is not given in advance, but has to be estimated from observations on the market. Since the size N of banking portfolios (i.e. the dimension of space in which the minimum is sought) is large and the number of observed data (the lengths T of the time series for the various assets) is always bounded, we never have a sufficient amount of information to reliably reconstruct the underlying stochastic process and estimate the risk functional. Our estimates of risk will therefore fluctuate from sample to sample, and the weights of the optimal portfolio will fluctuate along with them. This problem has long been known in finance, and a large number of noise reduction techniques have been developed to deal with it over the past decades, at least for the simple case of the variance. Variance is, however, not the only risk measure in use, and e.g. for fat tailed distributions it can even be grossly misleading. Noise reduction techniques for alternative risk measures are either much less developed or nonexistent.
We have performed an extensive numerical and analytical study of the unfiltered and unconstrained optimization problem for various risk measures (variance, mean absolute deviation, expected shortfall, and maximal loss) and various underlying processes (iid normal, correlated Gaussian, fat tailed, non-stationary GARCH-type) recently. We have found that the noise sensitivity strongly depends on the ratio N/T, and that there exists a critical value of N/T for all the above risk measures, beyond which the optimization is not feasible. This critical ratio is 1 for the variance and mean absolute deviation, ˝ for maximal loss, and a value, smaller than ˝ (and depending on the threshold beyond which the conditional average loss is calculated) for expected shortfall. Upon approaching this critical point the fluctuations in the estimation error of risk increase tremendously: the average error diverges with an exponent -1/2, the variance of its distribution with -1. The weights of the optimal portfolio for a given sample also show strong deviations from their exact values, with the variance and all the higher moments of their distribution diverging as one approaches the critical point.
The critical indices associated with these divergences are universal, i.e. independent of the structure of the market, the risk measure, and the character of the underlying fluctuations, whereas the prefactors of the scaling laws do depend on the covariance structure (predominantly positive correlations enhancing, negative ones decreasing the strength of the divergence).
When short selling is excluded (or any other constraint is applied that makes the domain over which the optimum is sought finite) sample to sample fluctuations can obviously not diverge any more. However, the instability will still be present, now manifesting itself through the strong sample to sample fluctuations of the weights, with an increasing number of them sticking to the boundaries defined by the constraints as we go deeper and deeper into the region N/T > 1 . Thus a ban on short selling leads to a spontaneous reduction of the portfolio size. Clearly, in these cases the solution is determined more by the constraints than by the objective function.
Similar remarks can be made also about the effect of filtering: unless the dimension of the problem is brutally reduced, the weights will keep jumping about thereby rendering any investment decision based on this optimization problem essentially illusory.
The instability of portfolio selection is related to the algorithmic phase transitions discovered recently.
As the structure of a large number of multidimensional regression and modelling problems is very similar to portfolio optimization, the scope of the above observations extends far beyond finance, and covers a large number of problems in operations research, machine learning, bioinformatics, medical science, economics, and technology.
This work has been supported by the ''Cooperative Center for Communication Networks Data Analysis'', a NAP project sponsored by the National Office of Research and Technology under grant No.\ KCKHA005.