はてなブックマーク

Reading Time: 3 minutes 検索拡張生成 (RAG) アプリケーションで、テキストだけでなく、表、グラフ、チャート、図など、さまざまな種類のデータを処理できれば、その有用性が飛躍的に高まります。そのためには、テキストや画像などの形式の情報を一貫して理解し、応答を生成することができるフレームワークが必要です。 この記事では、マルチモダリティ (複数の種類のデータ) を扱う上での課題と、マルチモーダル RAG パイプラインを構築するためのアプローチについて説明します。説明を簡潔にするために、画像とテキストの 2 つのモダリティのみに焦点を当てます。 マルチモダリティはなぜ難しいのでしょうか? 企業が扱う非構造化データは、フォルダーをいっぱいにしている高解像度の画像や、テキストが含まれる表、グラフ、図などが混在した PDF のように、複数のモダリティにまたがって散らばっている

Reading Time: 3 minutes ご注意: この記事は NeMo Framework のアップデートのため、2024 年 6 月 10 日に大幅に変更を加えました。 本記事では、NeMo Framework を使用して、日本語の大規模言語モデル (LLM) の PEFT (ファインチューニングの手法の一種)を実行する方法を説明します。 NeMo Framework とは NeMo Framework は、LLM をはじめ、生成 AI モデルを構築、カスタマイズするためのクラウドネイティブなフレームワークです。NGC 上にコンテナーが公開されており、すぐに利用を開始することができます。 NeMo Framework は、NVIDIA AI Enterprise の対象ソフトウェアになっているため、エンタープライズ サポートを希望される場合は、NVIDIA AI Enterpri

Debugging is difficult. Debugging across multiple languages is especially challenging, and debugging across devices often requires a team with varying skill sets and expertise to reveal the underlying problem. Yet projects often require using multiple languages, to ensure high performance where necessary, a user-friendly experience, and compatibility where possible. Unfortunately, there is no sing

Reading Time: 3 minutes GPU アプリケーションを高速化する方法には、主にコンパイラ指示行、プログラミング言語、ライブラリの 3 つがあります。OpenACC などは指示行ベースのプログラミング モデルで、コードをスムーズに GPU に移植し、高速化することができます。使い方は簡単ですが、特定のシナリオでは最適なパフォーマンスが得られない場合があります。 CUDA C や C++ などのプログラミング言語は、アプリケーションを高速化する際に、より大きな柔軟性を与えてくれます。しかし、最新のハードウェアで最適なパフォーマンスを実現するために、新しいハードウェア機能を活用したコードを書くことも、ユーザーの責任です。そこで、そのギャップを埋めるのが、ライブラリです。 コードの再利用性を高めるだけでなく、NVIDIA 数学ライブラリは、最大の性能向上のために GPU ハード

NVIDIA is now publishing Linux GPU kernel modules as open source with dual GPL/MIT license, starting with the R515 driver release. You can find the source code for these kernel modules in the NVIDIA/open-gpu-kernel-modules GitHub page This release is a significant step toward improving the experience of using NVIDIA GPUs in Linux, for tighter integration with the OS, and for developers to debug, i

PyTorch と NVIDIA TensorRT を新たに統合し、1 行のコードで推論を高速化する Torch-TensorRT に期待しています。PyTorch は、今では代表的なディープラーニング フレームワークであり、世界中に数百万人のユーザーを抱えています。TensorRT はデータ センター、組み込み、および車載機器で稼働する GPU アクセラレーションプラットフォーム全体で、高性能なディープラーニングの推論を行うための SDK です。この統合により、PyTorch ユーザーは TensorRT を使用する際、簡素化されたワークフローを通じて非常に高い推論性能を得ることができます。 Torch-TensorRT とは Torch-TensorRT は、TensorRT の推論最適化を NVIDIA GPU で利用するための PyTorch の統合ソフトウェアです。たった 1 行

WSL2 is available on Windows 11 outside the Windows Insider Preview. For more information about what is supported, see the CUDA on WSL User Guide. In June 2020, we released the first NVIDIA Display Driver that enabled GPU acceleration in the Windows Subsystem for Linux (WSL) 2 for Windows Insider Program (WIP) Preview users. At that time, it was still an early preview with a limited set of feature

Ray Tracing Gems II is now available to download for free via Apress. This Open Access book is a must-have for anyone interested in real-time rendering. Ray tracing is the holy grail of gaming graphics, simulating the physical behavior of light to bring real-time, cinematic-quality rendering to even the most visually intense games. Ray tracing is also a fundamental algorithm used for architecture

NVIDIA Riva for DevelopersNVIDIA® Riva is a set of GPU-accelerated multilingual speech and translation microservices for building fully customizable, real-time conversational AI pipelines. Riva includes automatic speech recognition (ASR), text-to-speech (TTS), and neural machine translation (NMT) and is deployable in all clouds, in data centers, at the edge, or in embedded devices. With Riva, orga

NVIDIA’s Deep Learning Institute (DLI) delivers practical, hands-on training and certification in AI at the edge for developers, educators, students, and lifelong learners. This is a great way to get the critical AI skills you need to thrive and advance in your career. You can even earn certificates to demonstrate your understanding of Jetson and AI when you complete these free, open-source course

NVIDIA Nsight™ Systems is a system-wide performance analysis tool designed to visualize an application’s algorithms, identify the largest opportunities to optimize, and tune to scale efficiently across any quantity or size of CPUs and GPUs, from large servers to our smallest systems-on-a-chip (SoCs).

Historically, accelerating your C++ code with GPUs has not been possible in Standard C++ without using language extensions or additional libraries: CUDA C++ requires the use of __host__ and __device__ attributes on functions and the triple-chevron syntax for GPU kernel launches. OpenACC uses #pragmas to control GPU acceleration. Thrust lets you express parallelism portably but uses language extens

WSL2 is available on Windows 11 outside the Windows Insider Preview. For more information about what is supported, see the CUDA on WSL User Guide. In response to popular demand, Microsoft announced a new feature of the Windows Subsystem for Linux 2 (WSL 2)—GPU acceleration—at the Build conference in May 2020. This feature opens the gate for many compute applications, professional tools, and worklo

WSL2 is available on Windows 11 outside of Windows Insider Preview. Please read the CUDA on WSL user guide for details on what is supported Microsoft Windows is a ubiquitous platform for enterprise, business, and personal computing systems. However, industry AI tools, models, frameworks, and libraries are predominantly available on Linux OS. Now all users of AI - whether they are experienced profe

Today, during the 2020 NVIDIA GTC keynote address, NVIDIA founder and CEO Jensen Huang introduced the new NVIDIA A100 GPU based on the new NVIDIA Ampere GPU architecture. This post gives you a look inside the new A100 GPU, and describes important new features of NVIDIA Ampere architecture GPUs. The diversity of compute-intensive applications running in modern cloud data centers has driven the expl

The new NVIDIA A100 GPU based on the NVIDIA Ampere GPU architecture delivers the greatest generational leap in accelerated computing. The A100 GPU has revolutionary hardware capabilities and we’re excited to announce CUDA 11 in conjunction with A100. CUDA 11 enables you to leverage the new hardware capabilities to accelerate HPC, genomics, 5G, rendering, deep learning, data analytics, data science

NVIDIA Makes 3D Deep Learning Research Easy with Kaolin PyTorch Library Research efforts in 3D computer vision and AI have been rising side-by-side like two skyscrapers. But the trip between these formidable towers has involved clambering up and down dozens of stairwells. To bridge that divide, NVIDIA recently released Kaolin, which in a few steps moves 3D models into the realm of neural networks.

GPUDirect Storage: A Direct Path Between Storage and GPU Memory As AI and HPC datasets continue to increase in size, the time spent loading data for a given application begins to place a strain on the total application’s performance. When considering end-to-end application performance, fast GPUs are increasingly starved by slow I/O. I/O, the process of loading data from storage to GPUs for process

All NVIDIA GPUs starting with the Kepler generation support fully-accelerated hardware video encoding, and all GPUs starting with Fermi generation support fully-accelerated hardware video decoding.  As of July 2019 Kepler, Maxwell, Pascal, Volta and Turing generation GPUs support hardware encoding, and Fermi, Kepler, Maxwell, Pascal, Volta and Turing generation GPUs support hardware decoding. The

Update: Jetson Nano and JetBot webinars. We’ve received a high level of interest in Jetson Nano and JetBot, so we’re hosting two webinars to cover these topics. The Jetson Nano webinar discusses how to implement machine learning frameworks, develop in Ubuntu, run benchmarks, and incorporate sensors. Register for the Jetson Nano webinar. A Jetbot webinar has Python GPIO library tutorials and inform

The NVIDIA Asteroids demo showcases how the mesh shading technology built into NVIDIA’s Turing GPU architecture can dramatically improve performance and image quality when rendering a substantial number of very complex objects in a scene. The following video highlights the capabilities of the mesh shader in the Asteroids demo. This video shows off the Asteroids demo in action. Turing introduces a

Imagine waiting for your flight at the airport. Suddenly, an important business call with a high profile customer lights up your phone. Tons of background noise clutters up the soundscape around you —  background chatter, airplanes taking off, maybe a flight announcement. You have to take the call and you want to sound clear. We all have been in this awkward, non-ideal situation. It’s just part of