Philosophy We strive to create an environment conducive to many different types of research across many different time scales and levels of risk. Learn more about our Philosophy Learn more
タグ検索の該当結果が少ないため、タイトル検索結果を表示しています。
Lyra: A New Very Low-Bitrate Codec for Speech Compression
Philosophy We strive to create an environment conducive to many different types of research across many different time scales and levels of risk. Learn more about our Philosophy Learn more
September 7, 2024 Hacker News Reddit Lobsters RussianI want to provide a smooth experience to my site visitors, so I work on accessibility and ensure it works without JavaScript enabled. I care about page load time because some pages contain large illustrations, so I minify my HTML. But one thing makes turning my blog light as a feather a pain in the ass. The hurdleSee, a major win in traffic redu
Compression Dictionary Transport の PHP 実装 - Web圧縮の次世代技術 - 弁護士ドットコム株式会社 Creators’ blog
はじめに 現代の Web アプリケーションにおいて、パフォーマンスはユーザー体験を左右する重要な要素です。これまで、gzip、Brotli、Zstandard といった優れた圧縮技術が、サーバーからクライアントへのデータ転送量を削減し、Web サイトの高速化に貢献してきました。 しかし、SPA（シングルページアプリケーション）や API 駆動型のアーキテクチャが主流となる中で、新たな課題が浮上しています。それは、数十 KB 程度の JSON データや動的に読み込まれる UI コンポーネントといった、比較的小さな HTTP レスポンスにおいて、圧縮効率が上がりにくいという問題です。既存の圧縮アルゴリズムは、データ内にある程度の繰り返しパターンが存在しなければ、その効果を十分に発揮できないという特性があるためです。 この課題を解決すべく登場したのが、新しい Web 標準仕様「Compressi
langchainのアップデートで追加された"Contextual Compression Retriever"を使用して,200ページ超のwebページを読みこませたQ&Aボットを作成する - Qiita
UnstructuredURLLoaderでホームページの内容を読みこみ、textsplitterで適切なサイズにテキストを分割します。 その後、OpenAIEmbeddingsを使用して、埋め込みを作成します。 その埋め込みをEmbeddingsFilterを用いてフィルタリングします。 (EmbeddingsFilter Making an extra LLM call over each retrieved document is expensive and slow. The EmbeddingsFilter provides a cheaper and faster option by embedding the documents and query and only returning those documents which have sufficiently simil
Compression Dictionary Transport (Shared Brotli) によるコンテンツ圧縮の最適化 | blog.jxck.io
Intro Chrome で Compression Dictionary Transport の Experiment が行われている。 Intent to Experiment: Compression dictionary transport with Shared Brotli https://groups.google.com/a/chromium.org/g/blink-dev/c/NgH-BeYO72E この提案の仕様および本サイトへの適用について解説する。 brotli の Dictionary 圧縮方式は、基本的に「同じ値が出てきたら、それらをまとめて小さく表現する」という方式が中心となる。 # 繰り返しを数値で表現する場合 from: aaaabbbbb to: a4b5 この方式は、対象としたデータの中で、如何に効率よく「同じ値」を見つけるかが肝となる。例えば以下の例
GitHub - jtesta/ssh-audit: SSH server & client security auditing (banner, key exchange, encryption, mac, compression, compatibility, security, etc)
usage: ssh-audit.py [-h] [-4] [-6] [-b] [-c] [-d] [-g <min1:pref1:max1[,min2:pref2:max2,...]> / <x-y[:step]>] [-j] [-l {info,warn,fail}] [-L] [-M custom_policy.txt] [-m] [-n] [-P "Built-In Policy Name" / custom_policy.txt] [-p N] [-T targets.txt] [-t N] [-v] [--conn-rate-test N[:max_rate]] [--dheat N[:kex[:e_len]]] [--get-hardening-guide platform] [--list-hardening-guides] [--lookup alg1[,alg2,...
GitHub - marv1nnnnn/llm-min.txt: Min.js Style Compression of Tech Docs for LLM Context
If you've ever used an AI coding assistant (like GitHub Copilot, Cursor, or others powered by Large Language Models - LLMs), you've likely encountered situations where they don't know about the latest updates to programming libraries. This knowledge gap exists because AI models have a "knowledge cutoff" – a point beyond which they haven't learned new information. Since software evolves rapidly, th
GitHub - mhx/dwarfs: A fast high-compression read-only file system for Linux, FreeBSD, macOS and Windows
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Introducing OpenZL: An Open Source Format-Aware Compression Framework
OpenZL is a new open source data compression framework that offers lossless compression for structured data. OpenZL is designed to offer the performance of a format-specific compressor with the easy maintenance of a single executable binary. You can get started with OpenZL today by visiting our Quick Start guide and the OpenZL GitHub repository. Learn more about the theory behind OpenZL in this wh
GitHub - deepseek-ai/DeepSeek-OCR: Contexts Optical Compression
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - google/lyra: A Very Low-Bitrate Codec for Speech Compression
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - phoboslab/qoi: The “Quite OK Image Format” for fast, lossless image compression
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Show navigation It is absolutely idiotic to have 64-bit pointers when I compile a program that uses less than 4 gigabytes of RAM. When such pointer values appear inside a struct, they not only waste half the memory, they effectively throw away half of the cache. – Donald Knuth (2008) Truer words have (almost) never been spoken. We also see CPU vendors not actually shipping 64-bit CPUs and Android
IntroStable Diffusion is currently inspiring the open source machine learning community as well as upsetting artists world wide. I was curious to see what else this impactful technology release could be used for other than making professional artists and designers ponder their job security. While experimenting with the model, I found that it makes for an extremely powerful lossy image compression
LangChainの新機能Contextual Compression Retrieverを試す｜mah_lab / 西見 公宏
Q&Aチャットボットのようなシステムを作成するとき、ユーザーの問い合わせに関連した情報をプロンプトに埋め込んで精度の高い回答を返す、といった仕組みはもはや一般的だと思います。 https://blog.langchain.dev/improving-document-retrieval-with-contextual-compression/その上で、関連情報を取り出す仕組みとしてベクトルDBの利用が一般的になってきていますが、抽出した文章が必ずしも質問に対して適切な情報源になっているとは限らない可能性はあります。類似度から算出して似ていると評価されていても、文脈が違うケースもあったりするのではないでしょうか。 先日（4/21）追加されたContextual Compression Retrieverはまさにこの問題を解決するためのもので、ベクトルDBなどから抽出した情報の評価を行い、更に
Reported Supply Chain Compromise Affecting XZ Utils Data Compression Library, CVE-2024-3094 | CISA
Official websites use .gov A .gov website belongs to an official government organization in the United States. Secure .gov websites use HTTPS A lock (A locked padlock) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Philosophy We strive to create an environment conducive to many different types of research across many different time scales and levels of risk. Learn more about our Philosophy Learn more
GitHub - phiresky/sqlite-zstd: Transparent dictionary-based row-level compression for SQLite
The data will be moved to _table_name_zstd, while table_name will be a view that can be queried as normally, including SELECT, INSERT, UPDATE, and DELETE queries. This function will not compress any data by itself, you need to call zstd_incremental_maintenance afterwards. config is a json object describing the configuration. See TransparentCompressConfig for detail. The following differences apply
RFC8879 TLS Certificate Compression について - ASnoKaze blog
「RFC8879 TLS Certificate Compression」が昨日公開されました。 これは、TLSハンドシェイク中に送信されるサーバ証明書を圧縮する仕組みを定義しています。 これによって、ハンドシェイクの通信量を削減できます。ハンドシェイク中は、パケットロスが起こっても後続のパケットは多くないのでロスリカバリとしては不利な状況です。ハンドシェイクに必要なパケット数が減るというのはメリットが有るのかなと思います。 また、QUIC(HTTP/3)においてもTLSハンドシェイクを利用していますが、QUICではClient Address Validationが終わるまで一度に送れる通信量に制限があるため、通信量が減ることはそういった意味でもメリットがあります。 この点については、FastlyのPatrick McManus氏がブログを書かれています。実際に送信されるパケット数がどう
GitHub - TheRealOrange/icer_compression: Progressive, error tolerant, wavelet-based image compression algorithm
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
How to Improve Rails Caching with Brotli Compression Updated Sep 12, 2023 7 minute read Caching is an effective way to speed up the performance of Rails applications. However, the costs of an in-memory cache database could become significant for larger-scale projects. In this blog post, I’ll describe optimizing the Rails caching mechanism using the Brotli compression algorithm instead of the defau
Large Text Compression Benchmark Matt Mahoney Last update: July 3, 2025. history This competition ranks lossless data compression programs by the compressed size (including the size of the decompression program) of the first 109 bytes of the XML text dump of the English version of Wikipedia on Mar. 3, 2006. About the test data. The goal of this benchmark is not to find the best overall compressi
CloudflareはSpeed Weekを通じて、パフォーマンスを最適化することの重要性について述べてきました。圧縮は、インターネット上で送信されるファイルサイズを小さくすることで、重要な役割を果たしています。ファイルサイズが小さくなると、ダウンロードが速くなり、Webサイトの読み込みが速くなり、ユーザーエクスペリエンスが向上します。 家庭用洗剤を実世界の例として挙げてみましょう。この洗剤は「一般的な洗剤のボトルは、90％が水で、実際の貴重な成分は10％以下である」と推定されます。一般的な500mlの家庭用洗剤の90％を取り除くと、重量は600gから60gに減ります。この削減は、使用者の受け取り時に水分を補給する指示とともに60gの小包だけを送付することを可能にします。ガロンに換算すると、この重量削減はビジネスにとって莫大な送料節約に直結し、環境への影響は言うまでもありません。 これが圧
Web Platform Dive into the web platform, at your pace.
Browser Image Compressionで画像をアップロード前に圧縮する[Javascript] | バシャログ。
どうもfujiharaです。本日は画像をアップロード前に圧縮できる BrowserImageCompressionをご紹介します。 インストール npm install browser-image-compression --save or yarn add browser-image-compression コード 以下が簡単な確認用コードになります。(react-create-appで作成) import './App.css'; import { useState } from 'react'; import imageCompression from 'browser-image-compression'; function App() { const [image_url, setImageUrl] = useState(''); const compressOption = {
![Browser Image Compressionで画像をアップロード前に圧縮する[Javascript] | バシャログ。](https://cdn-ak-scissors.b.st-hatena.com/image/square/6a7c7828e73c1691039a9aa0dce7f4c767e36c92/height=288;version=1;width=512/https%3A%2F%2Fbashalog.c-brains.jp%2Fimages%2Fcat_javascript.png)
LangChainにContextual Compressionという抽象化が追加されました。概要は以下にあります。 Contextual Compressionは「インデックスするドキュメントのテキスト」と「プロンプトに含めるコンテキストとしてのテキスト」の性質が異る点に注目して、ドキュメント検索の後処理としてプロンプトに含めるテキストの内容に変換処理をかけて改善します。 前提知識 「LLMに質問の答えを生成してもらうためにコンテキストとして事前に検索したテキストをプロンプトに挿入する」という大枠の仕組みさえ知っていればokです。 最近読んだ以下のスライドが分かりやすかったです。 使うRetriever ドキュメント取得のRetrieverにはChatGPT Retriever PluginsをLangChainでデバッグするで作ったRetriverを利用します。 「最近話題になった英語
GitHub - richardartoul/tsdb-layer: Time Series and FoundationDB. Millions of writes/s and 10x compression in under 2,000 lines of Go.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Compression Therapy Market Size, Trends, Growth Report 2035
Key Compression Therapy Market Trends Highlighted The Compression Therapy Market is experiencing notable trends driven by increasing awareness of chronic venous diseases and the growing prevalence of obesity and diabetes, which heighten the risk of related health conditions. The rising aging population globally has led to a higher demand for compression therapy products that address issues such as
Background In the late 19th to mid 20th century, the United States Department of Agriculture hired dozens of artists to paint watercolors of every fruit that grows in the United States. The collection contains 7,497 watercolor paintings, 87 line drawings, and 79 wax models created by approximately 21 artists. I thought this would make a great set of images to choose a random photo every day to use
GitHub - microsoft/LLMLingua: [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
![GitHub - microsoft/LLMLingua: [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.](https://cdn-ak-scissors.b.st-hatena.com/image/square/df5a4fa7b55867d2a9e4ef6d4ff10e7e51814a64/height=288;version=1;width=512/https%3A%2F%2Fopengraph.githubassets.com%2F64d810385db954fa0efb96e5e85bf3ad4e0a408dfc41650d5aa3d177365a525d%2Fmicrosoft%2FLLMLingua)
US Radial Artery Compression Devices Market Insights 2035
✉ info@marketresearchfuture.com 📞 +1 (855) 661-4441(US) 📞 +44 1720 412 167(UK) 📞 +91 2269738890(APAC) Global Market Outlook In-depth analysis of global and regional trends Analyze and identify the major players in the market, their market share, key developments, etc. To understand the capability of the major players based on products offered, financials, and strategies. Identify disrupting pro
Homogeneous Charge Compression Ignition Market Size, Share, Growth | Report, 2032
Global Market Outlook In-depth analysis of global and regional trends Analyze and identify the major players in the market, their market share, key developments, etc. To understand the capability of the major players based on products offered, financials, and strategies. Identify disrupting products, companies, and trends. To identify opportunities in the market. Analyze the key challenges in the
Compression and decompression in the browser with the Compression Streams API | Blog | Chrome for Developers
Build with Chrome Learn how Chrome works, participate in origin trials, and build with Chrome everywhere.
GitHub - Donaldcwl/browser-image-compression: Image compression in web browser
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens. This estimate is significantly smaller than currently available estimates in \cite{cover1978convergent}, \cite{lutati2023focus}. A natural byproduct is an algorithm for lossless compression of English text which com
sqlite-zstd: Transparent dictionary-based row-level compression for SQLite - An sqlite extension written in Rust to reduce the database size without losing functionality - phiresky's blog
data is a json blob from the perspective of SQLite, but it is parsed into a strictly typed structure in the Rust application code. After a year of collecting data, the SQLite database is already 7.6 GByte in size. Of course this is an extreme example of redundancy, but storing similar redundant data in databases is pretty common. SQLite doesn’t have any compression features, so of course I had to
The linear growth of key-value (KV) cache memory and quadratic computational in attention mechanisms complexity pose significant bottlenecks for large language models (LLMs) in long-context processing. While existing KV cache optimization methods address these challenges through token pruning or feature merging, they often incur irreversible information loss or require costly parameter retraining.
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
WebP: The WebPage compression format
Pointer compression in Oilpan · V8
Stable Diffusion Based Image Compression
Lyra: A New Very Low-Bitrate Codec for Speech Compression
How to Improve Rails Caching with Brotli Compression
Large Text Compression Benchmark
最大レベル11まで提供：Brotli圧縮とCompression Rulesの導入
Compression Streams are now supported on all browsers | Blog | web.dev
LangChainのContextual Compressionがどのようにコンテキストを圧縮しているのか
Rotation-Based Compression | Winston Cooke
AIDB on X: "Microsoftなどの研究者らが、プロンプト圧縮技術『LLMLingua-"2"』を公開し、タスクの精度を維持したまま圧縮率2-5倍を達成したと報告しています。 さらにプロンプトが圧縮されることにより、回答速度が倍速になるとのこと。 "LLMLingua-2: Learn Compression Target via Data Distillation for… https://t.co/4bEsVp3pfp"
GitHub - WICG/compression-dictionary-transport
LLMZip: Lossless Text Compression using Large Language Models
Conceptual compression means beginners don’t need to know SQL — hallelujah!
ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs
news.denfaminicogamer.jp
x.com
x.com
zenn.dev/somahc
megalodon.jp
megalodon.jp