MINE: Mutual Information Neural Estimation

We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size, trainable through back-prop, and strongly consistent. We present a handful of applications on which MINE can be

[elu_18]

相互情報量（MI）の推定に、KLダイバージェンスのDonsker-Varadhanh表現を使い，その表現中の尤度比のモデルにNNを使うことで，高次元のMIも高精度かつスケーラブルに求められる。エントロピーも推定でき、GAN，IBなどに使え