はてなブックマークアプリ

サクサク読めて、
アプリ限定の機能も多数!

アプリで開く

はてなブックマーク

  • はてなブックマークって?
  • アプリ・拡張の紹介
  • ユーザー登録
  • ログイン
  • Hatena

はてなブックマーク

トップへ戻る

  • 総合
    • 人気
    • 新着
    • IT
    • 最新ガジェット
    • 自然科学
    • 経済・金融
    • おもしろ
    • マンガ
    • ゲーム
    • はてなブログ(総合)
  • 一般
    • 人気
    • 新着
    • 社会ニュース
    • 地域
    • 国際
    • 天気
    • グルメ
    • 映画・音楽
    • スポーツ
    • はてな匿名ダイアリー
    • はてなブログ(一般)
  • 世の中
    • 人気
    • 新着
    • 新型コロナウイルス
    • 働き方
    • 生き方
    • 地域
    • 医療・ヘルス
    • 教育
    • はてな匿名ダイアリー
    • はてなブログ(世の中)
  • 政治と経済
    • 人気
    • 新着
    • 政治
    • 経済・金融
    • 企業
    • 仕事・就職
    • マーケット
    • 国際
    • はてなブログ(政治と経済)
  • 暮らし
    • 人気
    • 新着
    • カルチャー・ライフスタイル
    • ファッション
    • 運動・エクササイズ
    • 結婚・子育て
    • 住まい
    • グルメ
    • 相続
    • はてなブログ(暮らし)
    • 掃除・整理整頓
    • 雑貨
    • 買ってよかったもの
    • 旅行
    • アウトドア
    • 趣味
  • 学び
    • 人気
    • 新着
    • 人文科学
    • 社会科学
    • 自然科学
    • 語学
    • ビジネス・経営学
    • デザイン
    • 法律
    • 本・書評
    • 将棋・囲碁
    • はてなブログ(学び)
  • テクノロジー
    • 人気
    • 新着
    • IT
    • セキュリティ技術
    • はてなブログ(テクノロジー)
    • AI・機械学習
    • プログラミング
    • エンジニア
  • おもしろ
    • 人気
    • 新着
    • まとめ
    • ネタ
    • おもしろ
    • これはすごい
    • かわいい
    • 雑学
    • 癒やし
    • はてなブログ(おもしろ)
  • エンタメ
    • 人気
    • 新着
    • スポーツ
    • 映画
    • 音楽
    • アイドル
    • 芸能
    • お笑い
    • サッカー
    • 話題の動画
    • はてなブログ(エンタメ)
  • アニメとゲーム
    • 人気
    • 新着
    • マンガ
    • Webマンガ
    • ゲーム
    • 任天堂
    • PlayStation
    • アニメ
    • バーチャルYouTuber
    • オタクカルチャー
    • はてなブログ(アニメとゲーム)
    • はてなブログ(ゲーム)
  • おすすめ

    Google I/O

『ruder.io』

  • 人気
  • 新着
  • すべて
  • ML and NLP Research Highlights of 2021

    6 users

    www.ruder.io

    Credit for the title image: Liu et al. (2021) 2021 saw many exciting advances in machine learning (ML) and natural language processing (NLP). In this post, I will cover the papers and research areas that I found most inspiring. I tried to cover the papers that I was aware of but likely missed many relevant ones. Feel free to highlight them as well as ones that you found inspiring in the comments.

    • テクノロジー
    • 2022/01/25 19:46
    • 機械学習
    • あとで読む
    • Recent Advances in Language Model Fine-tuning

      3 users

      www.ruder.io

      Recent Advances in Language Model Fine-tuning This article provides an overview of recent methods to fine-tune large pre-trained language models. Fine-tuning a pre-trained language model (LM) has become the de facto standard for doing transfer learning in natural language processing. Over the last three years (Ruder, 2018), fine-tuning (Howard & Ruder, 2018) has superseded the use of feature extra

      • テクノロジー
      • 2021/02/25 14:51
      • Transformer
      • 機械学習
      • あとで読む
      • ML and NLP Research Highlights of 2020

        33 users

        www.ruder.io

        ML and NLP Research Highlights of 2020 This post summarizes progress in 10 exciting and impactful directions in ML and NLP in 2020. The selection of areas and methods is heavily influenced by my own interests; the selected topics are biased towards representation and transfer learning and towards natural language processing (NLP). I tried to cover the papers that I was aware of but likely missed m

        • テクノロジー
        • 2021/01/19 20:40
        • 機械学習
        • 自然言語処理
        • NLP
        • machine learning
        • 言語
        • あとで読む
        • *あとで読む
        • Why You Should Do NLP Beyond English

          4 users

          www.ruder.io

          Why You Should Do NLP Beyond English 7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English. Natural language processing (NLP) research predominantly focuses on developing methods that work well for English despite the many positive benefits of working on other languages. These benefits rang

          • テクノロジー
          • 2020/08/02 12:54
          • 10 Tips for Research and a PhD

            3 users

            www.ruder.io

            10 Tips for Research and a PhD This post outlines 10 things that I did during my PhD and found particularly helpful in the long run. This advice should be most relevant to people studying machine learning (ML) and natural language processing (NLP) as that is what I did in my PhD. Having said that, this advice is not just limited to PhD students. If you are an independent researcher, want to start

            • テクノロジー
            • 2020/05/28 09:09
            • 10 ML & NLP Research Highlights of 2019

              8 users

              www.ruder.io

              10 ML & NLP Research Highlights of 2019 This post gathers ten ML and NLP research directions that I found exciting and impactful in 2019. This post gathers ten ML and NLP research directions that I found exciting and impactful in 2019. For each highlight, I summarise the main advances that took place this year, briefly state why I think it is important, and provide a short outlook to the future. T

              • テクノロジー
              • 2020/01/07 08:17
              • 機械学習
              • あとで読む
              • The State of Transfer Learning in NLP

                5 users

                www.ruder.io

                The State of Transfer Learning in NLP This post expands on the NAACL 2019 tutorial on Transfer Learning in NLP. It highlights key insights and takeaways and provides updates based on recent work. Update 16.10.2020: Added Chinese and Spanish translations. This post expands on the NAACL 2019 tutorial on Transfer Learning in NLP. The tutorial was organized by Matthew Peters, Swabha Swayamdipta, Thoma

                • 学び
                • 2019/08/21 09:56
                • あとで読む
                • NAACL 2019 Highlights

                  3 users

                  www.ruder.io

                  NAACL 2019 Highlights This post discusses highlights of NAACL 2019. It covers transfer learning, common sense reasoning, natural language generation, bias, non-English languages, and diversity and inclusion. Update 19.04.20: Added a translation of this post in Spanish. This post discusses highlights of the 2019 Annual Conference of the North American Chapter of the Association for Computational Li

                  • テクノロジー
                  • 2019/06/14 09:23
                  • *あとで読む
                  • AAAI 2019 Highlights: Dialogue, reproducibility, and more

                    8 users

                    www.ruder.io

                    AAAI 2019 Highlights: Dialogue, reproducibility, and more This post discusses highlights of AAAI 2019. It covers dialogue, reproducibility, question answering, the Oxford style debate, invited talks, and a diverse set of research papers. This post discusses highlights of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19). I attended AAAI 2019 in Honolulu, Hawaii last week. Overa

                    • テクノロジー
                    • 2019/02/08 10:46
                    • 機械学習
                    • 10 Exciting Ideas of 2018 in NLP

                      11 users

                      www.ruder.io

                      10 Exciting Ideas of 2018 in NLP This post gathers 10 ideas that I found exciting and impactful this year—and that we'll likely see more of in the future. For each idea, it highlights 1-2 papers that execute them well. This post gathers 10 ideas that I found exciting and impactful this year—and that we'll likely see more of in the future. For each idea, I will highlight 1-2 papers that execute the

                      • テクノロジー
                      • 2018/12/21 07:14
                      • nlp
                      • 機械学習
                      • あとで読む
                      • Requests for Research

                        6 users

                        www.ruder.io

                        Requests for Research It can be hard to find compelling topics to work on and know what questions to ask when you are just starting as a researcher. This post aims to provide inspiration and ideas for research directions to junior researchers and those trying to get into research. This post aims to provide inspiration and ideas for research directions to junior researchers and those trying to get

                        • テクノロジー
                        • 2018/03/05 01:04
                        • 研究
                        • Optimization for Deep Learning Highlights in 2017

                          36 users

                          www.ruder.io

                          Optimization for Deep Learning Highlights in 2017 Different gradient descent optimization algorithms have been proposed in recent years but Adam is still most commonly used. This post discusses the most exciting highlights and most promising recent approaches that may shape the way we will optimize our models in the future. This post discusses the most exciting highlights and most promising direct

                          • テクノロジー
                          • 2017/12/04 08:24
                          • deep learning
                          • sgd
                          • deeplearning
                          • 機械学習
                          • Word embeddings in 2017: Trends and future directions

                            12 users

                            www.ruder.io

                            Word embeddings in 2017: Trends and future directions Word embeddings are an integral part of current NLP models, but approaches that supersede the original word2vec have not been proposed. This post focuses on the deficiencies of word embeddings and how recent approaches have tried to resolve them. This post discusses the deficiencies of word embeddings and how recent approaches have tried to res

                            • テクノロジー
                            • 2017/10/23 09:31
                            • nlp
                            • research
                            • Deep Learning for NLP Best Practices

                              93 users

                              www.ruder.io

                              Deep Learning for NLP Best Practices Neural networks are widely used in NLP, but many details such as task or domain-specific considerations are left to the practitioner. This post collects best practices that are relevant for most tasks in NLP. This post gives an overview of best practices relevant for most tasks in natural language processing. Update July 26, 2017: For additional context, the Ha

                              • テクノロジー
                              • 2017/07/26 09:03
                              • NLP
                              • Deep Learning
                              • DeepLearning
                              • 自然言語処理
                              • あとで読む
                              • An Overview of Multi-Task Learning for Deep Learning

                                16 users

                                www.ruder.io

                                An Overview of Multi-Task Learning in Deep Neural Networks Multi-task learning is becoming more and more popular. This post gives a general overview of the current state of multi-task learning. In particular, it provides context for current neural network-based methods by discussing the extensive multi-task learning literature. Note: If you are looking for a review paper, this blog post is also av

                                • テクノロジー
                                • 2017/05/30 10:54
                                • deep learning
                                • 機械学習
                                • multi task
                                • Transfer Learning - Machine Learning's Next Frontier

                                  25 users

                                  www.ruder.io

                                  Transfer Learning - Machine Learning's Next Frontier Deep learning models excel at learning from a large number of labeled examples, but typically do not generalize to conditions not seen during training. This post gives an overview of transfer learning, motivates why it warrants our application, and discusses practical applications and methods. This post gives an overview of transfer learning and

                                  • テクノロジー
                                  • 2017/03/25 11:33
                                  • 機械学習
                                  • Deep Learning
                                  • 論文
                                  • あとで読む
                                  • AI
                                  • A survey of cross-lingual word embedding models

                                    3 users

                                    www.ruder.io

                                    After the discussion of cross-lingual embedding models, we will additionally look into how to incorporate visual information into word representations, discuss the challenges that still remain in learning cross-lingual representations, and finally summarize which models perform best and how to evaluate them. Monolingual mapping Methods that employ monolingual mapping train monolingual word represe

                                    • 学び
                                    • 2016/12/03 16:18
                                    • 言語
                                    • ruder.io

                                      3 users

                                      www.ruder.io

                                      This post discusses Command R and Command R+, the top open-weights model on Chatbot Arena at the time of its release and highlights their RAG and multilingual capabilities.

                                      • テクノロジー
                                      • 2016/11/16 18:01
                                      • On word embeddings - Part 1

                                        7 users

                                        www.ruder.io

                                        On word embeddings - Part 1 Word embeddings popularized by word2vec are pervasive in current NLP applications. The history of word embeddings, however, goes back a lot further. This post explores the history of word embeddings in the context of language modelling. This post presents the most well-known models for learning word embeddings based on language modelling. Table of contents: A brief hist

                                        • テクノロジー
                                        • 2016/08/23 21:58
                                        • 機械学習
                                        • Approximating the Softmax for Learning Word Embeddings

                                          5 users

                                          www.ruder.io

                                          On word embeddings - Part 2: Approximating the Softmax The softmax layer is a core part of many current neural network architectures. When the number of output classes is very large, such as in the case of language modelling, computing the softmax becomes very expensive. This post explores approximations to make the computation more efficient. This post gives an overview of approximations that can

                                          • テクノロジー
                                          • 2016/08/13 23:37
                                          • 機械学習
                                          • An overview of gradient descent optimization algorithms

                                            77 users

                                            www.ruder.io

                                            An overview of gradient descent optimization algorithms Gradient descent is the preferred way to optimize neural networks and many other machine learning algorithms but is often used as a black box. This post explores how many of the most popular gradient-based optimization algorithms such as Momentum, Adagrad, and Adam actually work. This post explores how many of the most popular gradient-based

                                            • テクノロジー
                                            • 2016/01/20 12:05
                                            • SGD
                                            • machinelearning
                                            • machine learning
                                            • deep learning
                                            • Optimization
                                            • machine-learning
                                            • 機械学習

                                            このページはまだ
                                            ブックマークされていません

                                            このページを最初にブックマークしてみませんか?

                                            『ruder.io』の新着エントリーを見る

                                            キーボードショートカット一覧

                                            j次のブックマーク

                                            k前のブックマーク

                                            lあとで読む

                                            eコメント一覧を開く

                                            oページを開く

                                            はてなブックマーク

                                            • 総合
                                            • 一般
                                            • 世の中
                                            • 政治と経済
                                            • 暮らし
                                            • 学び
                                            • テクノロジー
                                            • エンタメ
                                            • アニメとゲーム
                                            • おもしろ
                                            • アプリ・拡張機能
                                            • 開発ブログ
                                            • ヘルプ
                                            • お問い合わせ
                                            • ガイドライン
                                            • 利用規約
                                            • プライバシーポリシー
                                            • 利用者情報の外部送信について
                                            • ガイドライン
                                            • 利用規約
                                            • プライバシーポリシー
                                            • 利用者情報の外部送信について

                                            公式Twitter

                                            • 公式アカウント
                                            • ホットエントリー

                                            はてなのサービス

                                            • はてなブログ
                                            • はてなブログPro
                                            • 人力検索はてな
                                            • はてなブログ タグ
                                            • はてなニュース
                                            • ソレドコ
                                            • App Storeからダウンロード
                                            • Google Playで手に入れよう
                                            Copyright © 2005-2025 Hatena. All Rights Reserved.
                                            設定を変更しましたx