satojkovicのブックマーク - はてなブックマーク

Self Supervised Representation Learning in NLP
An overview of self-supervised pretext tasks in Natural Language Processing While Computer Vision is making amazing progress on self-supervised learning only in the last few years, self-supervised learning has been a first-class citizen in NLP research for quite a while. Language Models have existed since the 90’s even before the phrase “self-supervised learning” was termed. The Word2Vec paper fro
satojkovic 2020/05/26
リンク
A Commit History of BERT and its Forks
I recently came across an interesting thread on Twitter discussing a hypothetical scenario where research papers are published on GitHub and subsequent papers are diffs over the original paper. Information overload has been a real probl em in ML with so many new papers coming every month. If you could represent a paper as a code diff, many papers could be compressed down to <50 lines :) The diff wo
satojkovic 2020/05/25
リンク
The Illustrated Self-Supervised Learning
A visual introduction to self-supervised learning methods for visual representations. I first got introduced to self-supervised learning in a talk by Yann Lecun, where he introduced the “cake analogy” to illustrate the importance of self-supervised learning. In the talk, he said: “If intelligence is a cake, the bulk of the cake is self-supervised learning, the icing on the cake is supervised learn
satojkovic 2020/02/29
Self supervised learning for computer vision
リンク
1