Streaming PCA: Matching Matrix Bernstein and Near-Optimal Finite Sample Guarantees for Oja's Algorithm

This work provides improved guarantees for streaming principle component analysis (PCA). Given $A_1, \ldots, A_n\in \mathbb{R}^{d\times d}$ sampled independently from distributions satisfying $\mathbb{E}[A_i] = \Sigma$ for $\Sigma \succeq \mathbf{0}$, this work provides an $O(d)$-space linear-time single-pass streaming algorithm for estimating the top eigenvector of $\Sigma$. The algorithm nearly

[elu_18]

1982年に提案されたOjaの学習則（Hebb則で重みが発散しないように正規化した単純な学習則）が、Top-1/k のオンライン主成分分析で収束が最も速いことが示された。初期化を工夫したOja++も提案。証明力がすごい https://t.co/1Xlfyd