The Surprising Behaviour of Distance Metrics in High Dimensions

0) IntroductionDistance metrics, like Euclidean, Manhattan, Minkowsky and so on, suffer a lot when we increase the number of dimensions (features) of our data. Its like they dilute and loose their meaning. The become unreliable. This is not very easy to understand, and it comes into conflict with the general understanding that more features equals to better Machine Learning models, which is far fr