タグ検索の該当結果が少ないため、タイトル検索結果を表示しています。
GitHub - onceupon/Bash-Oneliner: A collection of handy Bash One-Liners and terminal tricks for data processing and Linux system maintenance.
I am glad that you are here! I was working on bioinformatics a few years ago and was amazed by those single-word bash commands which are much faster than my dull scripts, time saved through learning command-line shortcuts and scripting. Recent years I am working on cloud computing and I keep recording those useful commands here. Not all of them is oneliner, but i put effort on making them brief an
こんにちは．株式会社ゆめみの Keeth こと桑原です．Twitter には #つぶやきProcessing という魅力的なタグがあり，毎日数々の美しい作品がこのタグを付けて投稿されています．これを眺めるだけでも一日中過ごせるくらいです（個人の感覚です）． 今日はその中でも特に度肝を抜かれ，かつ感動した作品が 2022/11/04 に投稿されておりましたので，個人の拙い能力で解析に挑戦してみました 💁 ※一部解析しきれていない部分がありますが，ご容赦いただけますと幸いです…何分勉強中の者でして… 作品 なにはともあれ今回対象の作品． 初めてみたときは思わず言葉を失いました．いや，これ twitter のツイートですのでかなり文字数も少ないんですよ！なのにこの表現ってどうなってんの！？と… これはかなり学びになると思い解析を試みようと思い立った次第です．それにしても美しい… ソースコード
This article was discussed on Hacker News. I recently learned of csvquote, a tool that encodes troublesome CSV characters such that unix tools can correctly process them. It reverses the encoding at the end of the pipeline, recovering the original input. The original implementation handles CSV quotes using the straightforward, naive method. However, there’s a better approach that is not only simpl
"複雑なデータ処理 × 静的サイト" を両立させる、楽をするRails運用 / A low-effort Rails workflow that combines “Complex Data Processing × Static Sites”
Kaigi on Rails 2025 Day 2 2025/09/27発表、"複雑なデータ処理 × 静的サイト" を両立させる、楽をするRails運用
ノードエディタ形式の画像処理ツール「Image-Processing-Node-Editor」 - Qiita
はじめに 趣味でノードエディタ形式の画像処理ツール「Image-Processing-Node-Editor」を作りました。 その紹介の記事です。中身にOpenCVガッツリ使っているからアドカレOKですよね。。。👀？ ガッツリ使っているという意味では、GUI部分の DearPyGui のほうがガッツリ使っているかもしれませんが🤔 「Image-Processing-Node-Editor」とは 以下のように、ノードを接続していくことで、処理結果を可視化しながら画像処理が行えるツールです。 以下のような特徴があります。 主要な処理は全てPython ※ライブラリ部分除く 各処理を可視化しながら画像処理が試せる 自作ノードの追加が容易 (だと信じている) 記事書くために見直していましたが、イマイチ複雑ですわ、、、😇 OSS (Apache 2.0ライセンス) デフォルトでいくつかのAI機
自作パケット処理系の性能測定と可視化&改善のPDCAを回して最強のパケット処理系の作り方を学ぼう / Let's Measure the Performance of Packet Processing System with Python Tools.
Pycon APAC 2023で登壇した時の資料です。 https://pretalx.com/pyconapac2023/talk/G3LDSG/ https://2023-apac.pycon.jp/timetable?id=G3LDSG
Ruby で Processing がブラウザ上からできる p5.rb を作りました - おんがえしの blog
ruby.wasm + p5.js の組み合わせです。ほとんどの API は移植したので大体同じことができると思います。 https://p5rb.ongaeshi.me/ https://p5rb.ongaeshi.me/examples に色々サンプル。 https://p5rb.ongaeshi.me/editor にオンラインエディタがあります。 使い方 p5.rb を HTML に読みこめばすぐに使えます。 <html> <head> <script src="https://cdn.jsdelivr.net/npm/ruby-3_2-wasm-wasi@next/dist/browser.script.iife.js"></script> <script src="https://cdn.jsdelivr.net/npm/p5@1.5.0/lib/p5.js"></script
tbsp - tree-based source-processing language tbsp is an awk-like language that operates on tree-sitter syntax trees. to motivate the need for such a program, we could begin by writing a markdown-to-html converter using tbsp and tree-sitter-md [0]. we need some markdown to begin with: # 1 heading content of first paragraph ## 1.1 heading content of nested paragraph for future reference, this markdo
Production RAG: what I learned from processing 5M+ documents
I've spent the last 8 months in the RAG trenches, I want to share what actually worked vs. wasted our time. We built RAG for Usul AI (9M pages) and an unnamed legal AI enterprise (4M pages). Langchain + Llamaindex We started out with youtube tutorials. First Langchain → Llamaindex. Got to a working prototype in a couple of days and were optimistic with the progress. We run tests on subset of the d
GitHub - facebookresearch/audiocraft: Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - nlp-with-transformers/notebooks: Jupyter notebooks for the Natural Language Processing with Transformers book
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
インテル、新デバイス「Infrastructure Processing Unit」（IPU）発表。スマートNICを拡張しストレージ仮想化やネットワーク仮想化処理などCPUからオフロード可能に
インテル、新デバイス「Infrastructure Processing Unit」（IPU）発表。スマートNICを拡張しストレージ仮想化やネットワーク仮想化処理などCPUからオフロード可能に インテルは、クラウド事業者や通信サービス事業者向けに、インフラストラクチャー・プロセシング・ユニット（IPU）を発表しました（インテルジャパンの発表）。 Introducing the Infrastructure Processing Unit. https://t.co/qro7AYYA9o #SixFiveSummit pic.twitter.com/LY9e6G7Yn2 — Intel News (@intelnews) June 14, 2021 IPUは、ネットワークカードをインテリジェントにしたスマートNICをさらに拡張したもの。これまでCPUで行われていたストレージ仮想化処理やネット
On the Way to Democratized Stream Processing: RisingWave’s Roadmap
On the Way to Democratized Stream Processing: RisingWave’s Roadmap Two months ago, we open-sourced RisingWave, a cloud-native streaming database. RisingWave is developed on the mission to democratize stream processing — to make stream processing simple, affordable, and accessible. You may check out our recent blog, document, and source code for more information about RisingWave.Rome was not built
AWS Lambda enhances event processing with provisioned mode for SQS event-source mapping | Amazon Web Services
AWS News Blog AWS Lambda enhances event processing with provisioned mode for SQS event-source mapping Today, we’re announcing the general availability of provisioned mode for AWS Lambda with Amazon Simple Queue Service (Amazon SQS) Event Source Mapping (ESM), a new feature that customers can use to optimize the throughput of their event-driven applications by configuring dedicated polling resource
Step Functions Distributed Map – A Serverless Solution for Large-Scale Parallel Data Processing | Amazon Web Services
AWS News Blog Step Functions Distributed Map – A Serverless Solution for Large-Scale Parallel Data Processing I am excited to announce the availability of a distributed map for AWS Step Functions. This flow extends support for orchestrating large-scale parallel workloads such as the on-demand processing of semi-structured data. Step Function’s map state executes the same processing steps for multi
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
みなさんこんにちは。機械学習チームのたかぱい（@takapy0210）です。 2021年もあと1ヶ月となりましたね。皆様いかがお過ごしでしょうか。 ...さて12月といえば、毎年恒例のアドベントカレンダーの季節ですね！ というわけで、2021年もコネヒト Advent Calendarが始まります！🎉 初日となる本エントリでは、機械学習チームで使用しているSageMaker*1の機能である、Processing*2について、活用事例とともにご紹介しようと思います。 目次 SageMaker Processingとは? SKLearnProcessor / PySparkProcessor Processor / ScriptProcessor これまでの課題感 SageMaker Processorの活用方法 SageMakerからECRにあるコンテナを指定してProcessor Job
GitHub - image-js/image-js: Image processing and manipulation in JavaScript
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Google Cloud、インテルと共同開発したASIC「Infrastructure Processing Unit（IPU）」採用を発表。FPGAベースのSoCでサーバ本体の処理をオフロード
Google Cloud、インテルと共同開発したASIC「Infrastructure Processing Unit（IPU）」採用を発表。FPGAベースのSoCでサーバ本体の処理をオフロード Google Cloudとインテルは、両社が共同開発したクラウド基盤用のASIC「Intel Infrastructure Processing Unit」（Intel IPU）のGoogle Cloudでの採用を明らかにしました。 Data center infrastructure that is more secure, flexible and performant.@GoogleCloud C3 VM is first to feature custom Intel IPUs and 4th Gen Intel #XeonScalable processors in private pr
New General Purpose, Compute Optimized, and Memory-Optimized Amazon EC2 Instances with Higher Packet-Processing Performance | Amazon Web Services
The r6in and r6idn instances are available in the US East (Ohio, N. Virginia), US West (Oregon), and Europe (Ireland) regions in On-Demand and Spot form. Savings Plans and Reserved Instances are available. Inside the Instances As you can probably guess from these specs and from the blog post that I wrote to launch the c6in instances, all of these new instance types have a lot in common. I’ll do a
GitHub - deepseek-ai/smallpond: A lightweight data processing framework built on DuckDB and 3FS.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - xavctn/img2table: img2table is a table identification and extraction Python Library for PDF and images, based on OpenCV image processing
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - binpash/pash: PaSh: Light-touch Data-Parallel Shell Processing
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Processing large JSON files in Python without running out of memory
Processing large JSON files in Python without running out of memory by Itamar Turner-Trauring Last updated 06 Jan 2023, originally created 14 Mar 2022 If you need to process a large JSON file in Python, it’s very easy to run out of memory. Even if the raw data fits in memory, the Python representation can increase memory usage even more. And that means either slow processing, as your program swaps
Deciphering language processing in the human brain through LLM representations
Philosophy We strive to create an environment conducive to many different types of research across many different time scales and levels of risk. Learn more about our Philosophy Learn more
GitHub - google-gemini/genai-processors: GenAI Processors is a lightweight Python library that enables efficient, parallel content processing.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
RubyKaigi 2022 - Fast data processing with Ruby and Apache Arrow #rubykaigi - 2022-09-13 - ククログ
株式会社クリアコード > ククログ > RubyKaigi 2022 - Fast data processing with Ruby and Apache Arrow #rubykaigi 関連リンク： スライド（Rabbit Slide Show） スライド（SlideShare） リポジトリー 内容 RubyKaigi Takeout 2021のRed ArrowのトークではRed Arrowを中心にできることをたくさん紹介しました。その発展形として今年は実際に使えそうな感じになっていることを紹介したかったので、高速データ処理機能にフォーカスすることにしました。が、採択されて資料を作り始めてみると「実際に使えそう」というには各機能の実装にもう少しブラッシュアップが必要なことがわかりました。なんと。。。 ということで、Apache Arrowを使って高速にデータ処理できる各種方法につい
Route to Workers, automate your email processing2022-05-13 Cloudflare Email Routing has quickly grown to a few hundred thousand users, and we’re incredibly excited with the number of feature requests that reach our product team every week. We hear you, we love the feedback, and we want to give you all that you’ve been asking for. What we don’t like is making you wait, or making you feel like your
GitHub - chenquan/arkflow: High-performance Rust stream processing engine, providing powerful data stream processing capabilities, supporting multiple input/output sources and processors.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
2024年度の Ruby アソシエーション開発助成金の成果発表会である「Ruby Association Activity Report」で発表した資料です。 採択いただいて開発した、2D レトロゲームエンジン Reight (R8) を紹介しています。 Reight レトロゲームエンジン - …
Processing Arrays non-destructively: `for-of` vs. `.reduce()` vs. `.flatMap()`
Processing Arrays non-destructively: for-of vs. .reduce() vs. .flatMap() In this blog post, we look at three ways of processing Arrays: The for-of loop The Array method .reduce() The Array method .flatMap() The goal is to help you choose between these features whenever you need to process Arrays. In case you don’t know .reduce() and .flatMap() yet, they will both be explained to you. In order to g
GitHub - NVIDIA/Cosmos: Cosmos is a world model development platform that consists of world foundation models, tokenizers and video processing pipeline to accelerate the development of Physical AI at Robotics & AV labs. Cosmos is purpose built for physica
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - cshum/imagor: Fast, secure image processing server and Go library, using libvips
imagor is a fast, secure image processing server and Go library. imagor uses one of the most efficient image processing libraries libvips with Go binding vipsgen — also available for low-level libvips operations directly in Go. It is typically 4-8x faster than using the quickest ImageMagick settings. imagor implements libvips streaming that facilitates parallel processing pipelines, achieving high
GitHub - Kazuhito00/Image-Processing-Node-Editor: 処理の検証や比較検討での用途を想定したノードエディターベースの画像処理アプリ(A node editor-based image processing application intended for use in processing verification and comparison studies)
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - anishathalye/semlib: Build data processing and data analysis pipelines that leverage the power of LLMs 🧠
Semlib is a Python library for building data processing and data analysis pipelines that leverage the power of large language models (LLMs). Semlib provides, as building blocks, familiar functional programming primitives like map, reduce, sort, and filter, but with a twist: Semlib's implementation of these operations are programmed with natural language descriptions rather than code. Under the hoo
AWS Step Functions launches large-scale parallel workflows for data processing and serverless applications
AWS Step Functions expands support for iterating and processing large sets of data such as images, logs and financial data in Amazon Simple Storage Service (Amazon S3), a cloud object storage service. AWS Step Functions is a visual workflow service capable of orchestrating over 10,000 API actions from over 220 AWS services to automate business processes and data processing workloads. Now, AWS Step
GitHub - HojiChar/HojiChar: The robust text processing pipeline framework enabling customizable, efficient, and metric-logged text preprocessing.
HojiChar provides a way to combine multiple arbitrary text processing tasks into a streamlined pipeline. The sequence of operations can be described declaratively, ensuring portability. HojiChar allows users to gather detailed statistical information from large amounts of text during processing. It enables management of any Python text processing tasks, providing a Command Line Interface (CLI) cap
Downstream Processing Market Size, Share, Trends, 2035
Downstream Processing Market Research Report By Application (Biopharmaceuticals, Food and Beverage, Water and Wastewater Treatment, Biofuels, Cosmetics), By Technique (Chromatography, Filtration, Ultrafiltration, Centrifugation, Precipitation), By Product (Reagents, Equipment, Consumables, Membranes, Filters), By End Use (Pharmaceutical, Biotechnology, Food Industry, Environmental, Cosmetic Indust
Originally posted: 2025-03-16. View source code for this page here. Over the past few years, I've found myself using DuckDB more and more for data processing, to the point where I now use it almost exclusively, usually from within Python. We're moving towards a simpler world where most tabular data can be processed on a single large machine1 and the era of clusters is coming to an end for all but
Efficiently processing batched data using parallelization in AWS Lambda | Amazon Web Services
AWS Compute Blog Efficiently processing batched data using parallelization in AWS Lambda This post is written by Anton Aleksandrov, Principal Solutions Architect, AWS Serverless Efficient message processing is crucial when handling large data volumes. By employing batching, distribution, and parallelization techniques, you can optimize the utilization of resources allocated to your AWS Lambda func
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
私が感動した Processing 製の作品のソースコードを解析してみた
Fast CSV processing with SIMD
tbsp - tree-based source-processing language
GitHub - catatsuy/purl: Streamlining Text Processing
機械学習プロジェクトにおけるSageMaker Processingの使い所 - コネヒト開発者ブログ
Route to Workers, automate your email processing
Processing Gem ベースの、2D レトロゲームエンジンの開発
Why DuckDB is my first choice for data processing
www.youtube.com
x.com
mainichi.jp
skkfield.hatenablog.com
share.google
heartne2025.hateblo.jp