In the last 7 years or so there has been quite a bit of work on parallel machine learning approaches, enough that I felt like a summary might be helpful both for myself and others. In each case, I put in the earliest known citation. If I missed something please comment. One basic dividing line between parallel approaches is single-machine vs. multi-machine. Multi-machine approaches offer the poten