No Free Lunch TheoremsBroadly speaking, there are two no free lunch theorems. One for supervised machine learning (Wolpert 1996) and one for search/optimization (Wolpert and Macready 1997). For an overview of the (no) free lunch and associated theorems, see David Wolpert’s What does dinner cost? No Free Lunch for Supervised Machine Learning Hume (1739–1740) pointed out that ‘even after the observation of the frequent o
