When it comes to neural network design, the trend in the past few years has pointed in one direction: deeper. Whereas the state of the art only a few years ago consisted of networks which were roughly twelve layers deep, it is now not surprising to come across networks which are hundreds of layers deep. This move hasn’t just consisted of greater depth for depths sake. For many applications, the mo