datasets / llm-jp-corpus-v2 · GitLab
2024年度 サイバーエージェント新卒社内研修の「データモデリング」の資料公開 | CyberAgent Developers Blog
協業リテールメディアdivでデータエンジニアをしている千葉です。 本日は、先日弊社内で実施をしたAI事業本部 新人研修の一部である「データモデリング」について記載をします。 同じく講師として登壇をした yassun7010 も「データベースの歴史」について、ブログとして公開をしているため、合わせて見ていただけると嬉しいです。 ※今回の記事作成に合わせて一部加筆修正をしています。 基幹系と情報系 今回の研修では、データモデリングを扱うシステムを 基幹系 情報系 に分けて説明をしています。 というのも基幹系と情報系では、そもそもデータの扱われ方やシステムの特性が異なります。 基幹系システムではOLTPと呼ばれる処理システムになっており、オンラインでかつリアルタイムにデータを追加更新します。そのため、重要となってくるのが多くのトランザクション（処理数）を正確にさばくことです。代表例としては銀行の
FineWeb: decanting the web for the finest text data at scale - a Hugging Face Space by HuggingFaceFW
Discover amazing ML apps made by the community
Introducing Claude 3.5 Sonnet
Today, we’re launching Claude 3.5 Sonnet—our first release in the forthcoming Claude 3.5 model family. Claude 3.5 Sonnet raises the industry bar for intelligence, outperforming competitor models and Claude 3 Opus on a wide range of evaluations, with the speed and cost of our mid-tier model, Claude 3 Sonnet. Claude 3.5 Sonnet is now available for free on Claude.ai and the Claude iOS app, while Clau
megagonlabs/roberta-long-japanese · Hugging Face
YAML Metadata Error: "datasets[0]" with value "mC4 Japanese" is not valid. If possible, use a dataset id from https://hf.co/datasets. roberta-long-japanese (jumanpp + sentencepiece, mC4 Japanese) This is the longer input version of RoBERTa Japanese model pretrained on approximately 200M Japanese sentences. max_position_embeddings has been increased to 1282, allowing it to handle much longer inputs
Real-time Whisper WebGPU - a Hugging Face Space by Xenova
Discover amazing ML apps made by the community
GenAI Handbook
William Brown @willccbb | willcb.com v0.1 (June 5, 2024) Introduction This document aims to serve as a handbook for learning the key concepts underlying modern artificial intelligence systems. Given the speed of recent development in AI, there really isn’t a good textbook-style source for getting up-to-speed on the latest-and-greatest innovations in LLMs or other generative models, yet there is an
GitHub - transitive-bullshit/agentic: AI agent stdlib that works with any LLM and TypeScript AI SDK.
GitHub - lllyasviel/Omost: Your image is almost there!
Omost is a project to convert LLM's coding capability to image generation (or more accurately, image composing) capability. The name Omost (pronunciation: almost) has two meanings: 1) everytime after you use Omost, your image is almost there; 2) the O mean "omni" (multi-modal) and most means we want to get the most out of it. Omost provides LLMs models that will write codes to compose image visual
GitHub - lllyasviel/Omost: Your image is almost there!
Omost is a project to convert LLM's coding capability to image generation (or more accurately, image composing) capability. The name Omost (pronunciation: almost) has two meanings: 1) everytime after you use Omost, your image is almost there; 2) the O mean "omni" (multi-modal) and most means we want to get the most out of it. Omost provides LLMs models that will write codes to compose image visual
xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
This paper introduces xRAG, an innovative context compression method tailored for retrieval-augmented generation. xRAG reinterprets document embeddings in dense retrieval--traditionally used solely for retrieval--as features from the retrieval modality. By employing a modality fusion methodology, xRAG seamlessly integrates these embeddings into the language model representation space, effectively
Training and Finetuning Embedding Models with Sentence Transformers v3
Additionally, you can use SequentialEvaluator to combine multiple evaluators into one, which can then be passed to the SentenceTransformerTrainer. If you don't have the necessary evaluation data but still want to track the model's performance on common benchmarks, you can use these evaluators with data from Hugging Face: EmbeddingSimilarityEvaluator with STSb The STS Benchmark (a.k.a. STSb) is a c
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
GitHub - shisa-ai/shaberi: Lightblue LLM Eval Framework: tengu, elyza100, ja-mtbench, rakuda
Intelligent RAG, Fetch Surrounding Chunks — Elastic Search Labs
GitHub - RUC-NLPIR/FlashRAG: ⚡FlashRAG: A Python Toolkit for Efficient RAG Research
GitHub - huggingface/text-clustering: Easily embed, cluster and semantically label text datasets
Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents
GitHub - huggingface/text-embeddings-inference: A blazing fast inference solution for text embeddings models
Elasticsearch retrievers - How to use search retrievers in Elasticsearch — Elastic Search Labs
delve
