Torch | Training and investigating Residual Nets

February 4, 2016 by Sam Gross and Michael Wilber The post was co-authored by Sam Gross from Facebook AI Research and Michael Wilber from CornellTech. In this blog post we implement Deep Residual Networks (ResNets) and investigate ResNets from a model-selection and optimization perspective. We also discuss multi-GPU optimizations and engineering best-practices in training ResNets. We finally compar

[catindog]

“Using the CuDNN Torch bindings, one can select the fastest convolution kernels by setting cudnn.fastest and cudnn.benchmark to true. This automatically benchmarks each possible algorithm on your GPU and chooses the fastest one. This sped up the time per-mini-batch by about 40% on a single GPU, bu