Deep Learningと画像認識� �～歴史・理論・実践～Several recent papers have explored self-supervised learning methods for vision transformers (ViT). Key approaches include: 1. Masked prediction tasks that predict masked patches of the input image. 2. Contrastive learning using techniques like MoCo to learn representations by contrasting augmented views of the same image. 3. Self-distillation methods like DINO that distill a teacher ViT into a st
