Spring Boot 3の新機能を使ってみよう！ 2からアップグレードする手順、Observability機能、ネイティブイメージ化｜ハイクラス転職・求人情報サイト アンビ（AMBI）
ハイクラス求人TOPIT記事一覧Spring Boot 3の新機能を使ってみよう！ 2からアップグレードする手順、Observability機能、ネイティブイメージ化 Spring Boot 3の新機能を使ってみよう！ 2からアップグレードする手順、Observability機能、ネイティブイメージ化 Javaの開発フレームワークであるSpringの最新バージョンとして、Spring Boot 3が2022年11月にリリースされました。この記事ではSpring Boot 2で書かれたサンプルコードをSpring Boot 3にアップグレードしながら、考慮点や新機能を体感していただきます。ヴイエムウェア株式会社の星野真知さんによる解説です。 Javaのエコシステム、その中でも世界で一番の人気を誇るのが（JetBrains社の調査によると）Spring FrameworkおよびSpring B
January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience with neural networks. This implementation is for educational purposes, so it's missing lots of features/improv
For many years I used the Julia programming language for transforming, cleaning, analyzing, and visualizing data, doing statistics, and performing simulations. I published a handful of open-source packages for things like signed distance fields, nearest-neighbor search, and Turing patterns (among others), made visual explanations of Julia concepts like broadcasting and arrays, and used Julia to ma
SARS-CoV-2 is associated with changes in brain structure in UK Biobank - Nature
The global pandemic of SARS-CoV-2 has now claimed millions of lives across the world. There has been an increased focus by the scientific and medical community on the effects of mild-to-moderate COVID-19 in the longer term. There is strong evidence for brain-related pathologies, some of which could be a consequence of viral neurotropism1,2,14 or virus-induced neuroinflammation3,4,5,15, including t
What We Learned from a Year of Building with LLMs (Part I)
It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. The pace of improvements in LLMs, coupled with a parade of demos on social media, will fuel an estimated $200B investment in AI by 2025. LLMs are also broadly accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into
Note: This blog post is also available as a documentation page on Transformers. Large Language Models (LLMs) such as GPT3/4, Falcon, and LLama are rapidly advancing in their ability to tackle human-centric tasks, establishing themselves as essential tools in modern knowledge-based industries. Deploying these models in real-world tasks remains challenging, however: To exhibit near-human text unders
Understanding math will make you a better engineer.So, I am writing the best and most comprehensive book about it. I'm interested Knowing the mathematics behind machine learning algorithms is a superpower. If you have ever built a model for a real-life problem, you probably experienced that familiarity with the details goes a long way if you want to move beyond baseline performance. This is especi
microgpt
This is a brief guide to my new art project microgpt, a single file of 200 lines of pure Python with no dependencies that trains and inferences a GPT. This file contains the full algorithmic content of what is needed: dataset of documents, tokenizer, autograd engine, a GPT-2-like neural network architecture, the Adam optimizer, training loop, and inference loop. Everything else is just efficiency.
Prompt Engineering
Date: March 15, 2023 | Estimated Reading Time: 21 min | Author: Lilian Weng Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empirical science and the effect of prompt engineering methods can vary a lot among models, thus requiring heavy experimentation a
DeepSeek has recently released DeepSeek v3, which is currently state-of-the-art in benchmark performance among open-weight models, alongside a technical report describing in some detail the training of the model. Impressively, they’ve achieved this SOTA performance by only using 2.8 million H800 hours of training hardware time—equivalent to about 4e24 FLOP if we assume 40% MFU. This is about ten t
CloudFront VPC オリジンで実現するマルチリージョンのアクティブ/アクティブ構成 | Amazon Web Services
Amazon Web Services ブログ CloudFront VPC オリジンで実現するマルチリージョンのアクティブ/アクティブ構成 はじめに 現代のデジタル社会において、組織はサイバーセキュリティの脅威に対する懸念を強めており、インフラストラクチャをより適切に保護する方法を積極的に模索しています。高度化するサイバー攻撃の増加と、より厳格になるデータ保護規制により、コンテンツ配信インフラのセキュリティ確保は企業にとって重要事項となっています。安全なコンテンツ配信ソリューションの必要性は、かつてないほど高まっています。 最近、Amazon CloudFront は CloudFront VPC オリジン のサポートを発表しました。これにより、顧客は CloudFront ディストリビューションからのみプライベートサブネット内のオリジンにルーティングできるようになりました。このアーキテ
Patterns for Building LLM-based Systems & Products [ llm engineering production 🔥 ] · 66 min read Discussions on HackerNews, Twitter, and LinkedIn “There is a large class of problems that are easy to imagine and build demos for, but extremely hard to make products out of. For example, self-driving: It’s easy to demo a car self-driving around a block, but making it into a product takes a decade.”
I was recently trying to convince a friend of mine that ChatGPT hasn't memorized every possible medical record, and that when she was passing her blood work results the model was doing pattern matching in ways that even OpenAI couldn't really foresee. She couldn't believe me, and I totally understand why. It's hard to accept that we invented a technology that we don't fully comprehend, and that ex
How can neural networks learn the rich internal representations required for difficult tasks such as recognizing objects or understanding language? Yoshua Bengio, Yann LeCun, and Geoffrey Hinton are recipients of the 2018 ACM A.M. Turing Award for breakthroughs that have made deep neural networks a critical component of computing. Research on artificial neural networks was motivated by the observa
Illustrating Reinforcement Learning from Human Feedback (RLHF)
This article has been translated to Chinese 简体中文 and Vietnamese đọc tiếng việt. Language models have shown impressive capabilities in the past few years by generating diverse and compelling text from human input prompts. However, what makes a "good" text is inherently hard to define as it is subjective and context dependent. There are many applications such as writing stories where you want creati
How a simple Linux kernel memory corruption bug can lead to complete system compromise
In this case, reallocating the object as one of those three types didn't seem to me like a nice way forward (although it should be possible to exploit this somehow with some effort, e.g. by using count.counter to corrupt the buf field of seq_file). Also, some systems might be using the slab_nomerge kernel command line flag, which disables this merging behavior. Another approach that I didn't look
Date: June 23, 2023 | Estimated Reading Time: 31 min | Author: Lilian Weng Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerfu
GitHub - diff-usion/Awesome-Diffusion-Models: A collection of resources and papers on Diffusion Models
DiffEnc: Variational Diffusion with a Learned Encoder Beatrix M. G. Nielsen, Anders Christensen, Andrea Dittadi, Ole Winther arXiv 2023. [Paper] 30 Oct 2023 Upgrading VAE Training With Unlimited Data Plans Provided by Diffusion Models Tim Z. Xiao, Johannes Zenn, Robert Bamler arXiv 2023. [Paper] 30 Oct 2023 Successfully Applying Lottery Ticket Hypothesis to Diffusion Model Chao Jiang, Bo Hui, Boha
Solving Quantitative Reasoning Problems With Language Models
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz∗, Anders Andreassen†, David Dohan†, Ethan Dyer†, Henryk Michalewski†, Vinay Ramasesh†, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur∗, Guy Gur-Ari∗, and Vedant Misra∗ Google Research Abstract Language models have achieved remarkable performance on a wide range of tasks that require
Attention Is Off By One
By Evan Miller July 24, 2023 About which one cannot speak, one must pass over in silence. –Wittgenstein Do you see the off-by-one error in this formula? \[ \textrm{Attention}(Q, K, V) = \textrm{softmax}\left(\frac{QK^T}{\sqrt{d}}\right)V \] The attention formula is the central equation of modern AI, but there’s a bug in it that has been driving me nuts the last week. I tried writing a serious-look
Happy New Year: GPT in 500 lines of SQL - EXPLAIN EXTENDED
Translations: Russian This year, the talk of the town was AI and how it can do everything for you. I like it when someone or something does everything for me. To this end, I decided to ask ChatGPT to write my New Year's post: "Hey ChatGPT. Can you implement a large language model in SQL?" "No, SQL is not suitable for implementing large language models. SQL is a language for managing and querying d
The Andrej Karpathy episode. Andrej explains why reinforcement learning is terrible (but everything else is much worse), why model collapse prevents LLMs from learning the way humans do, why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education. Watch on YouTube; listen on Apple Podcasts or Spotify
Blog
Hachi: An (Image) Search engine Only the dead have seen the end of war .. George Santayana For quite some time now, i have been working on and off on a fully self-hosted search engine, in hope to make it easier to search across Personal data in an end to end manner. Even as individuals, we are hoarding and generating more and more data with no end in sight. Such "personal" data is being stored fro
2011 was an important year in the history of psychology, especially social psychology. First, it became apparent that one social psychologist had faked results for dozens of publications (https://en.wikipedia.org/wiki/Diederik_Stapel). Second, a highly respected journal published an article with the incredible claim that humans can foresee random events in the future, if they are presented without
Generative Modeling by Estimating Gradients of the Data Distribution | Yang Song
Introduction Existing generative modeling techniques can largely be grouped into two categories based on how they represent probability distributions. likelihood-based models, which directly learn the distribution’s probability density (or mass) function via (approximate) maximum likelihood. Typical likelihood-based models include autoregressive models , normalizing flow models , energy-based mode
“Death of a Salesforce”: Why AI Will Transform the Next Generation of Sales Tech The battle between every startup and incumbent comes down to whether the startup gets distribution before the incumbent gets innovation. In sales tech, it’s easy to assume incumbents like Salesforce and Hubspot have the edge. First, they are embedded as “systems of record,” so sales leaders are loath to rip them out a
RAPIDS Forest Inference Library: Prediction at 100 million rows per second
IntroductionRandom forests (RF) and gradient-boosted decision trees (GBDTs) have become workhorse models of applied machine learning. XGBoost and LightGBM, popular packages implementing GBDT models, consistently rank among the most commonly used tools by data scientists on the Kaggle platform. We see similar interest in forest-based models in industry, where they are applied to problems ranging fr
What We’ve Learned From A Year of Building with LLMs – Applied LLMs
A practical guide to building successful LLM products, covering the tactical, operational, and strategic. It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. And they’re getting better and cheaper every year. Coupled with a parade of demos on social media, there will be an estimated $200B investment in AI
This was co-authored with Yao Yue This is a collection of information on severe (SEV-0 or SEV-1, the most severe incident classifications) incidents at Twitter that were at least partially attributed to cache from the time Twitter started using its current incident tracking JIRA (2012) to date (2022), with one bonus incident from before 2012. Not including the bonus incident, there were 6 SEV-0s a
AI Timelines via Cumulative Optimization Power: Less Long, More Short — LessWrong
The general trend is clear: larger lifetime compute enables systems of greater generality and capability. Generality and performance are both independently expensive, as an efficient general system often ends up requiring combinations of many specialist subnetworks. BNNs and ANNs both implement effective approximations of bayesian learning[29]. Net training compute then measures the total intra-li
17 types of similarity and dissimilarity measures used in data science. | Towards Data Science
The following article explains various methods for computing distances and showing their instances in our daily lives. Additionally, it… Various ML metrics. Inspired by Maarten Grootendorst. "There is no Royal Road to Geometry." – Euclid Quick note: Everything written and visualized has been created by the author unless it was specified. Illustrations and equations were generated using tools like
Migrating Critical Traffic At Scale with No Downtime — Part 2
Shyam Gala, Javier Fernandez-Ivern, Anup Rokkam Pratap, Devang Shah Picture yourself enthralled by the latest episode of your beloved Netflix series, delighting in an uninterrupted, high-definition streaming experience. Behind these perfect moments of entertainment is a complex mechanism, with numerous gears and cogs working in harmony. But what happens when this machinery needs a transformation?
research!rsc: Transparent Telemetry for Open-Source Projects (Transparent Telemetry, Part 1)
Russ Cox February 8, 2023 research.swtch.com/telemetry-intro How do software developers understand which parts of their software are being used and whether they are performing as expected? The modern answer is telemetry, which means software sending data to answer those questions back to a collection server. This post is about why I believe telemetry is important for open-source projects, and what
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision – PyTorch
Blog FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision Attention, as a core layer of the ubiquitous Transformer architecture, is a bottleneck for large language models and long-context applications. FlashAttention (and FlashAttention-2) pioneered an approach to speed up attention on GPUs by minimizing memory reads/writes, and is now used by most libraries to accelerat
The Annotated Diffusion Model
In this blog post, we'll take a deeper look into Denoising Diffusion Probabilistic Models (also known as DDPMs, diffusion models, score-based generative models or simply autoencoders) as researchers have been able to achieve remarkable results with them for (un)conditional image/audio/video generation. Popular examples (at the time of writing) include GLIDE and DALL-E 2 by OpenAI, Latent Diffusion
Keenadu the tablet conqueror and the links between major Android botnets
In April 2025, we reported on a then-new iteration of the Triada backdoor that had compromised the firmware of counterfeit Android devices sold across major marketplaces. The malware was deployed to the system partitions and hooked into Zygote – the parent process for all Android apps – to infect any app on the device. This allowed the Trojan to exfiltrate credentials from messaging apps and socia
Ilya Sutskever’s Top 30 Reading List The First Law of Complexodynamics The Unreasonable Effectiveness of Recurrent Neural Networks Understanding LSTM Networks Recurrent Neural Network Regularization Keeping Neural Networks Simple by Minimizing the Description Length of the Weights Pointer Networks ImageNet Classification with Deep Convolutional Neural Networks Order Matters: Sequence to Sequence f
「Google Colab」で「distilabel」を試したので、まとめました。 1. distilabel「distilabel」は、LLMを使用してLLM用のデータセットを作成するためのAI Feadback (AIF) フレームワークです。 ・LLMの最も一般的なライブラリ・APIとの統合 (HuggingFace Transformers、OpenAI、vLLMなど) ・Self-Instruct、Preferenceデータセットなどの複数のタスクに対応 ・データセットを Argillaにエクスポートすることで、データ探索とさらなるアノテーションが容易に 2. セットアップGoogle Colabでのセットアップ手順は、次のとおりです。 (1) パッケージのインストール。 # パッケージのインストール !pip install distilabel[openai,argilla]
Prenatal Screening Tests Devices Market Size, Report 2032
Prenatal Screening Tests Devices Market Research Repor: Information By Test (Alpha-Fetoprotein (AFP) Tests, Free Beta Human Chorionic Gonadotropin (hCG) Tests, Inhibin A Tests, Non-invasive prenatal testing (NIPT), Pregnancy Associated Plasma Protein A (PAPP-A) Tests, Total Human Chorionic Gonadotropin (hCG) Tests, and Unconjugated Estriol (uE3) Tests), And By Region (North America, Europe, Asia-P
Llama from scratch (or how to implement a paper without crying)
Llama from scratchI want to provide some tips from my experience implementing a paper. I'm going to cover my tips so far from implementing a dramatically scaled-down version of Llama for training TinyShakespeare. This post is heavily inspired by Karpathy's Makemore series, which I highly recommend. I'm only going to loosely follow the layout of their paper; while the formatting and order of sectio
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
GPT in 60 Lines of NumPy | Jay Mody
Why I no longer recommend Julia
Optimizing your LLM in production
The Roadmap of Mathematics for Machine Learning
How has DeepSeek improved the Transformer architecture?
Patterns for Building LLM-based Systems & Products
Building AI Products In The Probabilistic Era
Deep Learning for AI – Communications of the ACM
LLM Powered Autonomous Agents
Andrej Karpathy — AGI is still a decade away
Thinking Fast and Slow - Replicability-Index
“Death of a Salesforce”: Why AI Will Transform the Next Generation of Sales Tech | Andreessen Horowitz
A decade of major cache incidents at Twitter
Aman's AI Journal • Primers • Ilya Sutskever's Top 30
Google Colab で distilabel を試す｜npaka