This paper summarizes my lecture at the Machine Learning Summer School 2003, Tübingen. Abstract: This contribution presents an overview of the theoretical and practical aspects of the broad family of learning algorithms based on Stochastic Gradient Descent, including Perceptrons, Adalines, K-Means, LVQ, Multi-Layer Networks, and Graph Transformer Networks. Léon Bottou: Stochastic Learning, Advance