Pythonでデータ処理をしている際、numpyにはまらないごちゃごちゃした前処理があり、ちょっと遅いんだよなぁ。。。となること、ないでしょうか。 ルーチンになっている解析であれば高速化を頑張る意味がありそうですが、新しい解析を試行錯誤している最中など、わざわざ高速化のためのコードをガリガリ書いていくのは辛いぐらいのフェーズ、ないでしょうか。 こんなとき、私はJuliaを使っています。Juliaは特別な書き方をしなくても高速になる場合が多く、並列処理も簡単にできます。 julialang.org Julia、いいらしいが名前は聞いたことがあるけど使うまでには至ってない、という方がと思います。今まで使っているコードの資産を書き直すのは嫌ですよね。 しかし、JuliaにはPythonの資産を活かしつつ高速にデータ処理がするための道具がそろっています。 今回の記事はPythonとJuliaをいっ
レッドハットの杉村です。Ansible のテクニカルサポートをしています。 今回は以前のお問い合わせいただいた事例から、SSH (Secure Shell Protocol) について一つ紹介しようと思います。Ansible は Linux サーバを制御対象とするときは SSH で接続して処理を実行しますので、SSH の通信についてのトラブルは問題に直結します。 RHEL 8.6 + Ansible Core 2.13 で確認しています。 Ansible の基本的な動作原理 まずは Ansible はどうやって動いているのかというのを軽く振り返ってみます。 Ansible が動作するサーバをコントロールノード、制御対象をマネージドノードと呼びます。流れを大まかに説明しますと、この図のようになります。 ① YAMLで書かれたプレイブックからタスクごとに小さなプログラムを生成する ② ①で生成
January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience with neural networks. This implementation is for educational purposes, so it's missing lots of features/improv
February 05, 2024 · 9 mins · 1675 words Share on: X · HN Discussion on HackerNews. Last September I hopped on board with Wallapop as a Search Data Scientist and since then part of my work has been working with Solr, an open-source search engine based on Lucene. I’ve got the basics of how a search engine works, but I had this itch to understand it even better. So, I rolled up my sleeves and decided
Most of my readers are probably familiar with procedural programming, object-oriented programming (OOP), and functional programming (FP). The majority of top programming languages on all of the language popularity charts (like TIOBE) support all three to some extent. Even if a programmer avoided one or more of those three paradigms like the plague, they’re likely at least aware of them and what th
Node.js
Notable Changes Deprecations and Removals (SEMVER-MAJOR) fs: remove permissive rmdir recursive (Antoine du Hamel) #37216 (SEMVER-MAJOR) fs: runtime deprecate rmdir recursive option (Antoine du Hamel) #37302 (SEMVER-MAJOR) lib: runtime deprecate access to process.binding('http_parser') (James M Snell) #37813 (SEMVER-MAJOR) lib: runtime deprecate access to process.binding('url') (James M Snell) #377
What's new in Python 3.11?
What's new in Python 3.11?Built-in TOML support, better exceptions, and typing improvements. By Tushar·InsightsPython The first beta release of Python 3.11 is out, bringing some fascinating features for us to tinker with. This is what you can expect to see in 2022's release of Python later this year. Even better error messagesPython 3.10 gave us better error messages in various regards, but Python
Bringing the power of AI to Windows 11 – unlocking a new era of productivity for customers and developers with Windows Copilot and Dev Home
Bringing the power of AI to Windows 11 – unlocking a new era of productivity for customers and developers with Windows Copilot and Dev Home The team and I are pumped to be back at Build with the developer community this year. Over the last year, Windows has continued to see incredible growth fueled by Windows 11 adoption. In fact, one of the most exciting areas driving that growth for Windows has
It is December. That means annoying Christmas things are everywhere, including but not limited to the annual programming semi-competition known as Advent of Code. The problem with Advent of Code is that it is a waste of time. Most of the puzzles are in the realm of either string processing (somewhat applicable to programming), logic puzzles (not really applicable to most programming), or stupid go
agent loop �� �p�� You are Manus, an AI agent created by the Manus team. You excel at the following tasks: 1. Information gathering, fact-checking, and documentation 2. Data processing, analysis, and visualization 3. Writing multi-chapter articles and in-depth research reports 4. Creating websites, applications, and tools 5. Using programming to solve various problems beyond development 6. Variou
OBS Studioに関する情報メモを書いてる記事 ※「OBS Studioに関する個人的メモ」を移転しました。(現在は閲覧不可) 記事投稿日 2021年10月6日 本記事は文字数が非常に多いため、ブラウザの検索機能をご活用ください。 ブラウザ検索のショートカットキー ・Windows : Ctrl + F ・macOS : Command + F 見づらくて申し訳ありません。 将来的には内容を分割して投稿したいと考えていますが、分けても長くなってしまうため、当面はこのページにまとめています。 この記事は以下の環境を使用して作成しています。 ※Linux、特定のデバイスが無いと表示されないソースのことはメモしていません。 ■Windowsの場合 OBS Studio 31.1.2 (それ以前のバージョン、および開発版も含む) OS : Windows 11 Pro 64bit (バージョン
o1 pro + AIエンジニアにチャットで指示しながら、研究的なことをさせてみる ｜Kan Hatakeyama
はじめに自律的にプログラミングをしてくれるAIエンジニアをいい感じに動かせるようになってきたので、今日はChatGPT + devinで研究的なことをさせてみます。 自動研究といえば、昨年の夏に話題になった、Sakana AIのAIサイエンティストが有名です。 ただ、研究のネタを考えるのはまだあまり得意でない気がしたので、今回は適宜、そこはスマホで指示を出しながら、human in the loopで進めていきます。 最初のセットアップを除いて、チャットをするだけで、基本的な研究作業をこなせそう感じでした。 下準備: リポジトリを作ってdevinに登録するはじめに、パソコンを使って設定をします。このセクションの作業以降は、スマホがあればOKです。 githubでレポジトリを作り、一つだけ、開発方針に関するファイルを作っておきます。 DevelopmentPolycy.md 開発､コメントな
Infinite Mac: An Instant-Booting Quadra in Your Browser
tl;dr I’ve extended James Friend’s in-browser Basilisk II port to create a full-featured classic 68K Mac in your browser. You can see it in action at system7.app or macos8.app. For a taste, see also this screencast: Backstory It’s a golden age of emulation. Between increasing CPU power, WebAssembly, and retrocomputing being so popular The New York Times is covering it, it’s never been easier to re
Rust and Cosmopolitan Libc2023-11-01 Update: I found out that Rust decides system constants like EINVAL and SIGTERM at compile-time, which means that the portability of Rust executables is limited to x86_64-linux and aarch64-linux, the targets with which the executables are built. It’s good to know that Cosmo can be a viable target for Rust, and perhaps later we may use extern system constants in
Try Mastering Emacs for free! Are you struggling with the basics? Have you mastered movement and editing yet? When you have read Mastering Emacs you will understand Emacs. It’s that time again: there’s a new major version of Emacs and, with it, a treasure trove of new features and changes. Notable features include the formal inclusion of native compilation, a technique that will greatly speed up y
Faster virtual machines: Speeding up programming language execution - Mort's Ramblings
Date: 2023-01-15 Git: https://gitlab.com/mort96/blog/blob/published/content/00000-home/00015-fast-interpreters.md In this post, I hope to explore how interpreters are often implemented, what a "virtual machine" means in this context, and how to make them faster. Note: This post will contain a lot of C source code. Most of it is fairly simple C which should be easy to follow, but some familiarity w
Node.js
Notable changes Add support for externally shared js builtins By default Node.js is built so that all dependencies are bundled into the Node.js binary itself. Some Node.js distributions prefer to manage dependencies externally. There are existing build options that allow dependencies with native code to be externalized. This commit adds additional options so that dependencies with JavaScript code
Or if the package is only meant to be run on Node.js and wants to fallback to CJS on older versions that don't have require(esm): { "type": "module", "exports": { // On new version of Node.js, both require() and import get the ESM version "module-sync": "./index.js", // On older version of Node.js, where "module-sync" and require(esm) are // not supported, use the CJS version to avoid dual-package
Python behind the scenes #11: how the Python import system works
If you ask me to name the most misunderstood aspect of Python, I will answer without a second thought: the Python import system. Just remember how many times you used relative imports and got something like ImportError: attempted relative import with no known parent package; or tried to figure out how to structure a project so that all the imports work correctly; or hacked sys.path when you couldn
Plan 9 Desktop Guide
PLAN 9 DESKTOP GUIDE INDEX What is Plan 9? Limitations and Workarounds Connecting to Other Systems VNC RDP SSH 9P Other methods Porting Applications Emulating other Operating Systems Virtualizing other Operating Systems Basics Window Management Copy Pasting Essential Programs Manipulating Text in the Terminal Acme - The Do It All Application Multiple Workspaces Tiling Windows Plumbing System Admin
The Alkyne GC · mcyoung
Alkyne is a scripting language I built a couple of years ago for generating configuration blobs. Its interpreter is a naive AST walker1 that uses ARC2 for memory management, so it’s pretty slow, and I’ve been gradually writing a new evaluation engine for it. This post isn’t about Alkyne itself, that’s for another day. For now, I’d like to write down some notes for the GC I wrote3 for it, and more
2024-10-16, Version 23.0.0 (Current), @RafaelGSS We’re excited to announce the release of Node.js 23! Key highlights include: Enabling require(esm) by default for Node.js applications Removing support for Windows 32-bit systems Stabilizing the node --run command Enhancements to the test runner, including glob pattern support for coverage files Node.js 23 will replace Node.js 22 as the ‘Current’ re
As Elixir's Machine Learning (ML) ecosystem grows, many Elixir enthusiasts who wish to adopt the new machine learning libraries in their projects are stuck at a crossroads of wanting to move away from their existing ML stack (typically Python) while not having a clear path of how to do so. I would like to take some time to talk to WHY I believe now is a good time to start porting over Machine Lear
Logging C Function Calls
May 19th, 2022 @ justine's web page Logging C Functions One of my favorite features of the Cosmopolitan Libc runtime is its --ftrace flag that logs C function calls. It's the simplest system for debugging programs I've ever used and it surprises me that I found no evidence of someone having invented it before. Here's one of its most important use cases. Have you ever had you debugger stupified by
Red Hat Enterprise LinuxのHigh Availability Add-OnでActive/Standby構成のクラスターを組んでみた | DevelopersIO
参考: Amazon EC2 オンデマンド料金 どちらのインスタンスファミリーも、インスタンスサイズが大きくなればなるほど、Red Hat Enterprise Linux HAのオンデマンド料金はRHELのオンデマンド料金に近似していくことが分かります。 クラスター構成を組む必要がある場合、ユースケースにもよりますが、処理性能などの関係で大きめのインスタンスサイズを選択することが多いと思います。大きめのインスタンスサイズを選択したとしても、素のRHELからそこまで大きな追加課金が派生しないのは嬉しいポイントですね。 やってみた 検証環境 今回検証を行う環境は以下の通りです。 3つEC2インスタンスを用意したのはスプリッドブレイン対策です。 HA Add-Onでは、クォーラムデバイスによるスプリッドブレイン対策も可能ですが、今回は3ノード構成でクラスターを組んでみます。 クォーラムデバイス
Node.js
Notable Changes Add fetch API Adds experimental support to the fetch API. This adds a --experimental-fetch flag that installs the fetch, Request, Response and Headers globals. [76a229c4ff] - (SEMVER-MINOR) lib: add fetch (Michaël Zasso) #41749 Add stream methods [1ae648567a] - (SEMVER-MINOR) stream: add iterator helper find (linkgoron) #41849 [62e1a68077] - (SEMVER-MINOR) stream: add toArray (Benj
redbean 2.0 release notes
redbean is a webserver in a zip executable that runs on six operating systems. The basic idea is if you want to build a web app that runs anywhere, then you download the redbean.com file, put your .html and .lua files inside it using the zip command, and then you've got a hermetic app you can deploy and share. I introduced this web server about a year ago on Hacker News, where it became the third
Expert used ChatGPT-4o to create a replica of his passport in just 5 minutes bypassing KYC
SECURITY AFFAIRS MALWARE NEWSLETTER ROUND 41 | Security Affairs newsletter Round 519 by Pierluigi Paganini – INTERNATIONAL EDITION | China admitted its role in Volt Typhoon cyberattacks on U.S. infrastructure | Symbolic Link trick lets attackers bypass FortiGate patches, Fortinet warns | Attackers are exploiting recently disclosed OttoKit WordPress plugin flaw | Laboratory Services Cooperative dat
Flipping Pages: An analysis of a new Linux vulnerability in nf_tables and hardened exploitation techniques
This blogpost is the next instalment of my series of hands-on no-boilerplate vulnerability research blogposts, intended for time-travellers in the future who want to do Linux kernel vulnerability research. Specifically, I hope beginners will learn from my VR workflow and the seasoned researchers will learn from my techniques. In this blogpost, I'm discussing a bug I found in nf_tables in the Linux
Using AWS CodePipeline for deploying container images to AWS Lambda Functions | Amazon Web Services
AWS DevOps & Developer Productivity Blog Using AWS CodePipeline for deploying container images to AWS Lambda Functions AWS Lambda launched support for packaging and deploying functions as container images at re:Invent 2020. In the post working with Lambda layers and extensions in container images, we demonstrated packaging Lambda Functions with layers while using container images. This post will t
はじめての自然言語処理 ELECTRA（BERT の事前学習手法の改良）による固有表現抽出の検証 | オブジェクトの広場
今回は BERT における事前学習の改良手法である ELECTRA の検証です。ELECTRA はモデルサイズ、データ、計算量が同一条件であればオリジナルの BERT を凌ぐ性能とのことなので結果が楽しみなところです。事前学習をした後のファインチューニングは、いつも livedoor News Corpus の文書分類ばかりだったので、今回は固有表現抽出を試すことにしました。 1. はじめに 今回は BERT における事前学習の改良手法である ELECTRA 1 の検証です。 BERT に関しては 第3回 で取り上げていますが、トークン化が Sentencepiece である為、トークン単位での処理に難がありました2。今回は ELECTRA を試すにあたり、そのあたりの対応も入れ、 Megagon Labs さんから公開されている UD_Japanese-GSD v2.6-NE 3 を使っ
Why APL is a language worth knowing
“A language that doesn't affect the way you think about programming, is not worth knowing.”, by Alan J. Perlis. Why APL is a language worth knowing Alan Perlis, the computer scientist recipient of the first Turing award, wrote “A language that doesn't affect the way you think about programming, is not worth knowing.” ― Alan J. Perlis, 1982. Special feature: Epigrams on programming. ACM Sigplan Not
Lambda recently improved the cold start performance of container images by up to 15x, but this isn't the only reason you should use them. The tooling, ecosystem, and entire developer culture has moved to container images and you should too. January 2, 2024 When AWS Lambda first introduced support for container-based functions, the initial reactions from the community were mostly negative. Lambda i
はじめに データアナリティクス事業本部ビッグデータチームのyosh-kです。 今回はIICS CDIでリソースをGithubで管理し、チェックインした際にGithub ActionsでIICS JOBを自動テストすることが可能か検証していきたいと思います。 前提条件 以下の動画や記事などで紹介されていた内容を元に検証と実装を一部修正させていただきました。 Automated Deployment of IICS Assets- CI/CD using Informatica API's 事前に任意のマッピングタスク、マッピング、タスクフローを作成してあることとします。 また、IICS CDIとGithubでのソース管理が設定されていることとします。まだの場合は、以下のブログを参考にしてください。 Informatica Cloud Data IntegrationでGitHub上のリポジト
Today, we are pleased to announce a new advanced CUDA feature, CUDA Graphs, has been brought to PyTorch. Modern DL frameworks have complicated software stacks that incur significant overheads associated with the submission of each operation to the GPU. When DL workloads are strong-scaled to many GPUs for performance, the time taken by each GPU operation diminishes to just a few microseconds and, i
Node.js
Notable Changes [6349b1d673] - (SEMVER-MINOR) dns: add a cancel() method to the promise Resolver (Szymon Marczak) #33099 [9ce9b016e6] - (SEMVER-MINOR) events: add max listener warning for EventTarget (James M Snell) #36001 [8390f8a86b] - (SEMVER-MINOR) http: add support for abortsignal to http.request (Benjamin Gruenbaum) #36048 [9c6be3cc90] - (SEMVER-MINOR) http2: allow setting the local window s
自動操作を行う場合、id、name、classなどを利用して要素を指定することになるので、属性値をメモしておきましょう。 ５．同様にボタンについても属性を調べます。その結果は以下のようになります。 <input class="gNO89b" value="Google 検索" aria-label="Google 検索" name="btnK" type="submit" data-ved="0ahUKEwiC0u6iu4nlAhXwyIsBHWwTBHcQ4dUDCAo"> 属性 値 ここで調べた属性を利用して要素を特定して自動操作を行うことになります。。 また、今回はChromeでのやり方を紹介しましたが、他のブラウザでも同様のことが可能です。同じWebアプリケーションを使用していてもブラウザによって出力される内容が異なる可能性もあるので、自動操作を行うブラウザを使用して要素を調べるよ
March 24, 2024 · 9 min · 1915 words · Robin Kåveland | Suggest Changes I worked for a few years in the intersection between data science and software engineering. On the whole, it was a really enjoyable time and I’d like to have the chance to do so again at some point. One of the least enjoyable experiences from that time was to deal with big CSV exports. Unfortunately, this file format is still v
Python 3, Pygame, and Debian Bookworm on the Miyoo A30 | jtolio.com
First published: Feb 1, 2025, 2:26pm -0400 Last edited: Feb 1, 2025, 8:48pm -0400 What if I told you you can get a full Linux computer with WiFi and a GPU in a small, pocket form factor for about $30 USD on AliExpress? You might say “but what about the new tariffs?” and I’d say “oof, oh right, okay who knows, but it’s probably still pretty cheap!” (Edit: looks like you can get one for about $40 no
v2022: Austin Huang, Suraj Subramanian, Jonathan Sum, Khalid Almubarak, and Stella Biderman. Original: Sasha Rush. The Transformer has been on a lot of people’s minds over the last year five years. This post presents an annotated version of the paper in the form of a line-by-line implementation. It reorders and deletes some sections from the original paper and adds comments throughout. This docume
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
JuliaとPythonを併用したデータ処理のススメ - MNTSQ Techブログ
Ansible の SSH の通信をデバッグする - 赤帽エンジニアブログ
GPT in 60 Lines of NumPy | Jay Mody
A search engine in 80 lines of Python
Implementing Logic Programming
Advent of Code on the Nintendo DS
Manus tools and prompts
OBS Studio に関するメモ - すたいるのOBS情報メモブログ
Rust and Cosmopolitan Libc | Blog Needs a Name
What's New in Emacs 28.1?
Node.js — Node.js v22.10.0 (Current)
Node.js — Node.js v23.0.0 (Current)
From Python to Elixir Machine Learning
The case for containers on Lambda (with benchmarks)
【IICS】GitHub Actionsを使用したIICS JOB自動テスト | DevelopersIO
Accelerating PyTorch with CUDA Graphs – PyTorch
Webアプリケーションを自動で操作してみよう | へっぽこ実験ブログ
Friends don't let friends export to CSV
The Annotated Transformer