Note: This blog post is also available as a documentation page on Transformers. Large Language Models (LLMs) such as GPT3/4, Falcon, and LLama are rapidly advancing in their ability to tackle human-centric tasks, establishing themselves as essential tools in modern knowledge-based industries. Deploying these models in real-world tasks remains challenging, however: To exhibit near-human text unders
Heya. Sorry for not writing for so long. I’ll make up for it with 3000 pages here. I’m just hopping right now. That’s kinda the only way to get me to blog anymore. I’ve rewritten this post so many times. It’s about AI. But AI is changing so fast that the post is out of date within a few days. So screw it. I’m busting this version out in one sitting. (Spoiler alert: There’s some Sourcegraph stuff a
Patterns for Building LLM-based Systems & Products [ llm engineering production 🔥 ] · 66 min read Discussions on HackerNews, Twitter, and LinkedIn “There is a large class of problems that are easy to imagine and build demos for, but extremely hard to make products out of. For example, self-driving: It’s easy to demo a car self-driving around a block, but making it into a product takes a decade.”
GitHub - RUC-NLPIR/FlashRAG: ⚡FlashRAG: A Python Toolkit for Efficient RAG Research (WWW2025 Resource)
[25/08/06] 🎯 NEW! We have added support for Reasoning Pipeline, which is a new paradigm that combines reasoning ability and retrieval, representing work that includes R1-Searcher, Search-R1,.... We evaluate the performance of the pipeline on various RAG benchmarks, it can achieve F1 scores close to 60 on multi hop inference datasets such as HotpotQA. See it in result table. [25/03/21] 🚀 Major Up
The State of Python 2025: Trends and Survey Insights | The PyCharm Blog
This is a guest post from Michael Kennedy, the founder of Talk Python and a PSF Fellow. Welcome to the highlights, trends, and key actions from the eighth annual Python Developers Survey. This survey is conducted as a collaborative effort between the Python Software Foundation and JetBrains’ PyCharm team. The survey results provide a comprehensive look at Python usage statistics and popularity tre
とうとうRyzen + RADEONのAMD環境にもWindowsで動くStable Diffusionがきた - 徒労日記
やっと来たAMD最初の一歩。DirectMLでAMD GPUを動かす 定期的にAMDｘSD x Winのことを調べていますが、今回この記事に行き当たりました。Running Stable Diffusion on Windows with an AMD GPU Unfortunately, in its current state, it relies on Nvidia’s CUDA framework, which means that it only works out of the box if you’ve got an Nvidia GPU. Fear not, however. Because Stable Diffusion is both a) open source and b) good, it has seen an absolute flurry of activ
Tiny Core Linux 13.0 is a full Linux desktop in 22 MB #Linux
February 11, 2022 AT 9:40 am Tiny Core Linux 13.0 is a full Linux desktop in 22 MB #Linux Most Linux-based operating systems are fairly large — usually well over a full gigabyte in size for the base install image. Tiny Core Linux, which clocks in at a svelte 22 MB download, just released version 13. It’s fast. And another neat trick: “Unlike most operating systems, the Tiny Core can run completel
The Andrej Karpathy episode. Andrej explains why reinforcement learning is terrible (but everything else is much worse), why model collapse prevents LLMs from learning the way humans do, why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education. Watch on YouTube; listen on Apple Podcasts or Spotify
Transformer models: an introduction and catalog — 2023 Edition January 16, 2023 52 minute read This post is now an ArXiV paper that you can print and cite. Update 05/2023 Another pretty large update after 4 months. I was invited to submit the article to a journal, so I decided to enlist some help from some LinkedIn colleages and completely revamp it. First off, we added a whole lot of new models,
連載「3分プロトタイピング」 Streamlitを用いたAIチャットアプリ RAGを使ってAIチャットアプリケーションに知識を与える ベクトルデータベース超入門(この記事です) 前回、前々回とAIアプリケーションのプロトタイプを作る時に便利な2つのフレームワーク: StreamlitとLlamaIndexを紹介しました。 この記事では、本格的なAIアプリケーションを作成するときに必要になることの多い、ベクトルデータベースを紹介します。今回も説明が長くなりますが、コード部分は3分で試せることを目指しています！ ベクトルデータベース、ベクトル検索とは ベクトルデータベースとはどのような技術か、AWSのドキュメントがわかりやすく説明しているので引用します。 ベクトルデータベースは、ベクトルを高次元の点として保存および取得する機能を提供します。 これらには、N 次元空間の最も近い近傍を効率的かつ高
The Best GPUs for Deep Learning in 2023 — An In-depth Analysis
Deep learning is a field with intense computational requirements, and your choice of GPU will fundamentally determine your deep learning experience. But what features are important if you want to buy a new GPU? GPU RAM, cores, tensor cores, caches? How to make a cost-efficient choice? This blog post will delve into these questions, tackle common misconceptions, give you an intuitive understanding
For demonstration purposes, we'll fine-tune the multilingual version of the small checkpoint with 244M params (~= 1GB). As for our data, we'll train and evaluate our system on a low-resource language taken from the Common Voice dataset. We'll show that with as little as 8 hours of fine-tuning data, we can achieve strong performance in this language. 1{}^11 The name Whisper follows from the acronym
Note: the performance scores shown in the table below have been updated to account for the new methodology introduced in November 2023, which added new benchmarks. More details in this post. Demo You can easily try the 13B Llama 2 Model in this Space or in the playground embedded below: To learn more about how this demo works, read on below about how to run inference on Llama 2 models. Inference I
Intended Uses Primary use cases The model is intended for broad commercial and research use in English. The model provides uses for general purpose AI systems and applications with visual and text input capabilities which require memory/compute constrained environments; latency bound scenarios; general image understanding; OCR; chart and table understanding. Our model is designed to accelerate res
Mastering Customer Segmentation with LLM | Towards Data Science
Unlock advanced customer segmentation techniques using LLMs, and improve your clustering models with advanced techniques Content Table · Intro · Data · Method 1: Kmeans · Method 2: K-Prototype · Method 3: LLM + Kmeans · Conclusion Intro A customer segmentation project can be approached in multiple ways. In this article I will teach you advanced techniques, not only to define the clusters, but to a
Technology Trends for 2024
This has been a strange year. While we like to talk about how fast technology moves, internet time, and all that, in reality the last major new idea in software architecture was microservices, which dates to roughly 2015. Before that, cloud computing itself took off in roughly 2010 (AWS was founded in 2006); and Agile goes back to 2000 (the Agile Manifesto dates back to 2001, Extreme Programming t
Machine Learning with PyTorch and Scikit-Learn has been a long time in the making, and I am excited to finally get to talk about the release of my new book. Initially, this project started as the 4th edition of Python Machine Learning. However, we made so many changes to the book that we thought it deserved a new title to reflect that. So, what’s new, you may wonder? In this post, I am excited to
トークナイザを使わない自然言語処理モデルである ByT5 と Charformer のご紹介です。従来の自然言語処理では多くの場合で文章を単語（あるいはサブワード）単位に分かち書きして処理しましたが、今回のモデルは直接、生のテキストを処理します。それでは実際に動かして単語（サブワード）ベースのモデルと比較してみましょう。 1. はじめに 今回は今年5月と6月に発表された ByT51 と Charformer2 の紹介をしたいと思います。一本の記事で 2 つのモデルを扱うのは、この連載では珍しいのですが、この二つはよく似ているというか、Charformer は 「ByT5 にもう一工夫加えたもの」くらいの認識なので、一度にさばいてしまいましょうということで。 さて、この二つのモデルの特徴ですが「分かち書きをしない」という点に尽きます。 今まで、この連載では BERT や T5 等の Tran
Accelerating Generative AI with PyTorch: Segment Anything, Fast – PyTorch
Blog Accelerating Generative AI with PyTorch: Segment Anything, Fast This post is the first part of a multi-series blog focused on how to accelerate generative AI models with pure, native PyTorch. We are excited to share a breadth of newly released PyTorch performance features alongside practical examples of how these features can be combined to see how far we can push PyTorch native performance.
When Open Becomes Opaque: The Changing Face of Open-Source Hardware Companies
July 12, 2023 AT 1:00 pm When Open Becomes Opaque: The Changing Face of Open-Source Hardware Companies Over the last 15+ years, innovative electronics companies have designed and released thousands of open-source hardware designs, creating a flourishing industry. Open-source hardware companies collectively created, and signed the open-source hardware definition which means products meet a uniform
GitHub - taishi-i/awesome-ChatGPT-repositories: A curated list of resources dedicated to open source GitHub repositories related to ChatGPT and OpenAI API
awesome-chatgpt-api - Curated list of apps and tools that not only use the new ChatGPT API, but also allow users to configure their own API keys, enabling free and on-demand usage of their own quota. awesome-chatgpt-prompts - This repo includes ChatGPT prompt curation to use ChatGPT better. awesome-chatgpt - Curated list of awesome tools, demos, docs for ChatGPT and GPT-3 awesome-totally-open-chat
今回は T5X と Prompt Tuning の検証をしてみました。T5X は JAX と Flax で実装された T5 の新世代実装です。 Prompt Tuning は近年流行している事前学習済みモデルとプロンプトで下流タスクを解く手法の一つです。 Prompt Tuning に関しては T5X で実装されたコードが公開されていたので、合わせて検証してみることにしました。 1. はじめに 今回は T5X1 と Prompt Tuning2 の検証とご紹介になります。 T5X は第7回で紹介した T53 の次世代実装になります。T5 は、Mesh Tensorflow4 を採用することで、 単一の TPU や GPU に全パラメータが格納できない大規模モデルを実現していますが、学習ループ周辺の実装は Tensorflow 1.x 系列の Estimator API を用いた、やや古びた
【要約】 DiffBIRは，画像復元の手法で，低品質な画像を高品質に復元する．Windows上でのインストールでは，TritonのインストールとDiffBIRのセットアップを行う．学習済みモデルを使用して，画像復元を行う．詳細は公式GitHubページで説明されている． 元画像と処理結果 【目次】 前準備 DiffBIR のインストール（Windows 上） DiffBIR の動作確認（Windows 上） DiffBIR を使う Python プログラムの実行（Windows 上） DiffBIR DiffBIRは画像復元の手法の一つである。 画像復元は、低品質または劣化した画像を元の高品質な状態に修復するタスクである。 このタスクでは、ノイズや歪みなどの複雑な問題に対処する必要がある。 DiffBIRは、2つの主要なステージから成り立っている。 最初のステージでは、画像復元が行われ、低品
We’re releasing an open source system, based on FSDP and QLoRA, that can train a 70b model on two 24GB GPUs. Summary Today, we’re releasing Answer.AI’s first project: a fully open source system that, for the first time, can efficiently train a 70b large language model on a regular desktop computer with two or more standard gaming GPUs (RTX 3090 or 4090). This system, which combines FSDP and QLoRA,
Welcome to Lesson 5 of 12 in our free course series, LLM Twin: Building Your Production-Ready AI Replica. You’ll learn how to use LLMs, vector DVs, and LLMOps best practices to design, train, and deploy a production ready “LLM twin” of yourself. This AI character will write like you, incorporating your style, personality, and voice into an LLM. For a full overview of course objectives and prerequi
Build and deploy ML inference applications from scratch using Amazon SageMaker | Amazon Web Services
Artificial Intelligence Build and deploy ML inference applications from scratch using Amazon SageMaker As machine learning (ML) goes mainstream and gains wider adoption, ML-powered inference applications are becoming increasingly common to solve a range of complex business problems. The solution to these complex business problems often requires using multiple ML models and steps. This post shows y
DisclaimerThese are my own views, not necessarily those of my employer. SummaryLarge language models (LLMs) represent a major advance in artificial intelligence (AI), and in particular toward the goal of human-like artificial general intelligence (AGI). It’s sometimes claimed, though, that machine learning is “just statistics”, hence that progress in AI is illusory with regard to this grander ambi
v2022: Austin Huang, Suraj Subramanian, Jonathan Sum, Khalid Almubarak, and Stella Biderman. Original: Sasha Rush. The Transformer has been on a lot of people’s minds over the last year five years. This post presents an annotated version of the paper in the form of a line-by-line implementation. It reorders and deletes some sections from the original paper and adds comments throughout. This docume
GitHub - ComfyUI-Workflow/awesome-comfyui: A collection of awesome custom nodes for ComfyUI
ComfyUI-Gemini_Flash_2.0_Exp (⭐+172): A ComfyUI custom node that integrates Google's Gemini Flash 2.0 Experimental model, enabling multimodal analysis of text, images, video frames, and audio directly within ComfyUI workflows. ComfyUI-ACE_Plus (⭐+115): Custom nodes for various visual generation and editing tasks using ACE_Plus FFT Model. ComfyUI-Manager (⭐+113): ComfyUI-Manager itself is also a cu
Paul: Today’s spotlight: Paolo Perrone, master of turning tech into scroll-stopping content. This one’s packed, let’s go 👀 ↓ I’m going to be honest with you. Most AI agent tutorials are garbage. They show you how to copy-paste LangChain code, build a demo that breaks the moment you try anything real, and leave you feeling like you learned something. Three months later, you try to build something
Sentiment analysis is the automated process of tagging data according to their sentiment, such as positive, negative and neutral. Sentiment analysis allows companies to analyze data at scale, detect insights and automate processes. In the past, sentiment analysis used to be limited to researchers, machine learning engineers or data scientists with experience in natural language processing. However
今回は教師なしの文章ベクトル化手法である MixCSE の検証です。教師なし学習ですから教師ありの手法よりは精度的に不利でしょうが、局面によっては役に立つケースもあるのでは？と試してみることに。公開されているコードは transformers ベースなのですが、今回は Colab の TPU で動かしてみたので、その方法も紹介しますね。 1. はじめに 今回は教師なしの文章ベクトル化手法である MixCSE1 の検証をしてみました。 本連載では文章ベクトル化のモデルとして、 Sentence BERT を取り上げたこと(第9回, 第18回)がありますが、品質の良いベクトルを生成する為には大量かつ良質の教師データが必要でした。 法律や特許のような特定領域に特化した文章を扱う局面では、対象領域の文書で学習したモデルを使いたいところですが、特定領域限定の都合良いデータはなかなか手に入りません。そ
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Optimizing your LLM in production
Cheating is All You Need | Sourcegraph Blog
Patterns for Building LLM-based Systems & Products
Andrej Karpathy — AGI is still a decade away
Transformer models: an introduction and catalog — 2023 Edition
3分プロトタイピング: ベクトルデータベース超入門 - ROUTE06 Tech Blog
Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers
Llama 2 is here - get it on Hugging Face
microsoft/Phi-3-vision-128k-instruct · Hugging Face
Machine Learning with PyTorch and Scikit-Learn
はじめての自然言語処理 ByT5 と Charformer の検証 | オブジェクトの広場
はじめての自然言語処理 T5X と Prompt Tuning の検証 | オブジェクトの広場
画像復元（DiffBIR，Python，PyTorch を使用）（Windows 上）
You can now train a 70b language model at home – Answer.AI
The 4 Advanced RAG Algorithms You Must Know to Implement
Do large language models understand us?
The Annotated Transformer
The Realistic Guide to Mastering AI Agents in 2026
Getting Started with Sentiment Analysis using Python
はじめての自然言語処理 MixCSE による教師なし文章ベクトル生成 | オブジェクトの広場