🪆 Introduction to Matryoshka Embedding Models

In this blogpost, we will introduce you to the concept of Matryoshka Embeddings and explain why they are useful. We will discuss how these models are theoretically trained and how you can train them using Sentence Transformers. Additionally, we will provide practical guidance on how to use Matryoshka Embedding models and share a comparison between a Matryoshka embedding model and a regular embeddi

[secondlife]

768次元なら、例えば0:64,0:128などの先頭の次元だけでもlossが低くなるように学習させる。なのでそこだけ使うことで、embeddingsの次元を低く使いたい時は使うことができる。