"genetic algorithms"の検索結果1 - 1 件 / 1件

• あとで読む

  GitHub - lucidrains/lion-pytorch: 🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

  • 5 users
  • github.com/lucidrains
  • テクノロジー
  • 2023/02/16

  Learning rate and weight decay: the authors write in Section 5 - Based on our experience, a suitable learning rate for Lion is typically 3-10x smaller than that for AdamW. Since the effective weight decay is lr * λ, the value of decoupled weight decay λ used for Lion is 3-10x larger than that for AdamW in order to maintain a similar strength. The initial value, peak value, and end value in the lea

  • PyTorch
  • 機械学習
  • あとで読む