The Probability of Backtest Overfitting

Authors

David H. Bailey, University of California, Davis
Jonathan Borwein, University of Newcastle - Australia
Marcos Lopez de Prado, Cornell University
Qiji Jim Zhu, Western Michigan UniversityFollow

Document Type

Article

Publication Date

4-2017

Abstract

Many investment firms and portfolio managers rely on backtests (ie, simulations of performance based on historical market data) to select investment strategies and allocate capital. Standard statistical techniques designed to prevent regression overfitting, such as hold-out, tend to be unreliable and inaccurate in the context of investment backtests. We propose a general framework to assess the probability of backtest overfitting (PBO). We illustrate this framework with specific generic, model-free and nonparametric implementations in the context of investment simulations; we call these implementations combinatorially symmetric cross-validation (CSCV). We show that CSCV produces reasonable estimates of PBO for several useful examples.

Comments

Download from the SSRN.

DOI: 10.21314/JCF.2016.322

WMU ScholarWorks Citation

Bailey, David H.; Borwein, Jonathan; Lopez de Prado, Marcos; and Zhu, Qiji Jim, "The Probability of Backtest Overfitting" (2017). Math Faculty Publications. 42.
https://scholarworks.wmich.edu/math_pubs/42