Python multiprocessing vs threading vs asyncio - JX通信社エンジニアブログ
エンジニアの鈴木（泰）です。 今回は、multiprocessingとthreadingとasyncioの違いとはなんだろう？という問に挑戦してみたいと思います。 この問の答えをグーグル先生に聞いてみると、非常にたくさんの情報がヒットします。しかしながら、どの情報も断片的なものばかりで（本記事もそうなのかもしれません）、色々と本を読んだりネットを漁ったりして、情報を補完しなければなりませんでした。 本記事は、僕が調べた限りの情報を集約し、この問に対する結論を１つの記事にまとめたものとなっています。 前提 マルチプロセスとは マルチスレッドとは Pythonにおけるマルチスレッド 本題 マルチプロセス（multiprocessingライブラリ）を利用したほうが良い場合 cpu_sec.py cpu_multiprocessing.py cpu_threading.py cpu_asyncio
Developers are increasingly relying on AI coding assistants to accelerate our daily workflows. These tools can autocomplete functions, suggest bug fixes, and even generate entire modules or MVPs. Yet, as many of us have learned, the quality of the AI’s output depends largely on the quality of the prompt you provide. In other words, prompt engineering has become an essential skill. A poorly phrased
January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience with neural networks. This implementation is for educational purposes, so it's missing lots of features/improv
TL;DR; We are changing std::sort in LLVM’s libcxx. That’s a long story of what it took us to get there and all possible consequences, bugs you might encounter with examples from open source. We provide some benchmarks, perspective, why we did this in the first place and what it cost us with exciting ideas from Hyrum’s Law to reinforcement learning. All changes went into open source and thus I can
Agents
Intelligent agents are considered by many to be the ultimate goal of AI. The classic book by Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (Prentice Hall, 1995), defines the field of AI research as “the study and design of rational agents.” The unprecedented capabilities of foundation models have opened the door to agentic applications that were previously unimaginabl
Piccolo - A Stackless Lua Interpreter 2024-05-01 History of piccolo A "Stackless" Interpreter Design Benefits of Stackless Cancellation Pre-emptive Concurrency Fuel, Pacing, and Custom Scheduling "Symmetric" Coroutines and coroutine.yieldto The "Big Lie" Rust Coroutines, Lua Coroutines, and Snarfing Zooming Out piccolo is an interpreter for the Lua language written in pure, mostly safe Rust with a
Golang Mini Reference 2022: A Quick Guide to the Modern Go Programming Language (REVIEW COPY)
Golang Mini Reference 2022 A Quick Guide to the Modern Go Programming Language (REVIEW COPY) Harry Yoon Version 0.9.0, 2022-08-24 REVIEW COPY This is review copy, not to be shared or distributed to others. Please forward any feedback or comments to the author. • feedback@codingbookspress.com The book is tentatively scheduled to be published on September 14th, 2022. We hope that when the release da
Join a VS Code Dev Days event near you to learn about AI-assisted development in VS Code. Update 1.54.1: The update addresses an issue with an extension dependency. Update 1.54.2: The update addresses these issues. Update 1.54.3: The update addresses this issue. Downloads: Windows: x64 Arm64 | Mac: Universal Intel silicon | Linux: deb rpm tarball Arm snap Welcome to the February 2021 release of Vi
Patterns for Building LLM-based Systems & Products [ llm engineering production 🔥 ] · 66 min read Discussions on HackerNews, Twitter, and LinkedIn “There is a large class of problems that are easy to imagine and build demos for, but extremely hard to make products out of. For example, self-driving: It’s easy to demo a car self-driving around a block, but making it into a product takes a decade.”
Tier 4 Support § Support for these targets is entirely experimental. If this target is provided by LLVM, LLVM may have the target as an experimental target, which means that you need to use Zig-provided binaries for the target to be available, or build LLVM from source with special configure flags. zig targets will display the target if it is available. This target may be considered deprecated by
It is December. That means annoying Christmas things are everywhere, including but not limited to the annual programming semi-competition known as Advent of Code. The problem with Advent of Code is that it is a waste of time. Most of the puzzles are in the realm of either string processing (somewhat applicable to programming), logic puzzles (not really applicable to most programming), or stupid go
I want to write a post about Pitchfork, explaining where it comes from, why it is like it is, and how I see its future. But before I can get to that, I think I need to share my mental model on a few things, in this case, Ruby’s GVL. For quite a long time, it has been said that Rails applications are mostly IO-bound, hence Ruby’s GVL isn’t that big of a deal and that has influenced the design of so
The Go Programming Language and Environment – Communications of the ACM
Go is a programming language created at Google in late 2007 and released as open source in November 2009. Since then, it has operated as a public project, with contributions from thousands of individuals and dozens of companies. Go has become a popular language for building cloud infrastructure: Docker, a Linux container manager, and Kubernetes, a container deployment system, are core cloud techno
Update 1.86.2: The update addresses these issues. Update 1.86.1: The update addresses these issues. Downloads: Windows: x64 Arm64 | Mac: Universal Intel silicon | Linux: deb rpm tarball Arm snap Welcome to the January 2024 release of Visual Studio Code. There are many updates in this version that we hope you'll like, some of the key highlights include: Per-window zoom levels - Adjust the zoom leve
Tier 4 Support § Support for these targets is entirely experimental. If this target is provided by LLVM, LLVM may have the target as an experimental target, which means that you need to use Zig-provided binaries for the target to be available, or build LLVM from source with special configure flags. zig targets will display the target if it is available. This target may be considered deprecated by
Python behind the scenes #13: the GIL and its effects on Python multithreading
As you probably know, the GIL stands for the Global Interpreter Lock, and its job is to make the CPython interpreter thread-safe. The GIL allows only one OS thread to execute Python bytecode at any given time, and the consequence of this is that it's not possible to speed up CPU-intensive Python code by distributing the work among multiple threads. This is, however, not the only negative effect of
State of the Common Lisp ecosystem, 2020 🎉 - Lisp journey
NEW: 9 videos (86min) about CLOS on my Common Lisp course. Out of 7h+ of content. Rated 4.7/5. Learn more and stay tuned. 🎥 I also have cool Lisp showcases on Youtube . The last ones: how to build a web app in Common Lisp, part 1 and 2. This is a description of the Common Lisp ecosystem, as of January, 2021, from the perspective of a user and contributor. The purpose of this article is both to gi
Engineering for Slow Internet How to minimize user frustration in Antarctica. Hello everyone! I got partway through writing this post while I was still in Antarctica, but I departed before finishing it. I’m going through my old draft posts, and I found that this one was nearly complete. It’s a bit of a departure from the normal content you’d find on brr.fyi, but it reflects my software / IT engine
prompts.chat
Welcome to the “Awesome ChatGPT Prompts” repository! While this collection was originally created for ChatGPT, these prompts work great with other AI models like Claude, Gemini, Hugging Face Chat, Llama, Mistral, and more. ChatGPT is a web interface created by OpenAI that provides access to their GPT (Generative Pre-trained Transformer) language models. The underlying models, like GPT-4o and GPT-o
We recently overhauled our internal tools for visualizing the compilation of JavaScript and WebAssembly. When SpiderMonkey’s optimizing compiler, Ion, is active, we can now produce interactive graphs showing exactly how functions are processed and optimized. You can play with these graphs right here on this page. Simply write some JavaScript code in the test function and see what graph is produced
Try Mastering Emacs for free! Are you struggling with the basics? Have you mastered movement and editing yet? When you have read Mastering Emacs you will understand Emacs. It’s that time again: there’s a new major version of Emacs and, with it, a treasure trove of new features and changes. Notable features include the formal inclusion of native compilation, a technique that will greatly speed up y
vim9script4pythondevelopers.md Vim9 script for Python Developers Vim9 script�Vim script��������������������������������������������������系��� def������義����������Vim script��vim9script�����使����������(vim9script���
Reflection is something a lot of people wish the Rust language had: It is not hard to stumble across somebody with an interesting use case for it. People want to use it for serialization, GCs, better interop, and so, so much more. If you can think of a task, there is somebody out there wishing they could implement it using reflection. Sadly, it does not look like it is coming any time soon. Still,
Plan 9 Desktop Guide
PLAN 9 DESKTOP GUIDE INDEX What is Plan 9? Limitations and Workarounds Connecting to Other Systems VNC RDP SSH 9P Other methods Porting Applications Emulating other Operating Systems Virtualizing other Operating Systems Basics Window Management Copy Pasting Essential Programs Manipulating Text in the Terminal Acme - The Do It All Application Multiple Workspaces Tiling Windows Plumbing System Admin
While getting my degree in Physics, I had to take classes in both MatLab and Python for scientific computing. I preferred python, where we used the SciPy and NumPy packages. In fact, I used those packages again (along with matplotlib) in an undergraduate research project simulating bacteria films. There's a catch: I was also pursuing a degree in Computer Science, and Python just wasn't fast enough
Following up from the last post, there is a lot more we need to cover. This was intended to be the post where we talk exclusively about benchmarks and numbers. But, I have unfortunately been perfectly taunted and status-locked, like a monster whose “aggro” was pulled by a tank. The reason, of course, is due to a few folks taking issue with my outright dismissal of the C and C++ APIs (and not showi
[PLAY WEB VERSION: ALONE] [PLAY WEB VERSION: WITH A FRIEND] [PRINT] [RULES] [SIMILAR PROJECTS] [SYMBOLS] [CREDITS] [CONTACT] [GAME HELPER ESP32 | MOBILE] [ASSEMBLY GUIDE] THE GAME I made this game to teach my daughter how buffer overflows work. My favorite part of comuting is looking at programs as something you can play with, and poke and twist and make it whatever you want. When your microwave o
Flipping Pages: An analysis of a new Linux vulnerability in nf_tables and hardened exploitation techniques
This blogpost is the next instalment of my series of hands-on no-boilerplate vulnerability research blogposts, intended for time-travellers in the future who want to do Linux kernel vulnerability research. Specifically, I hope beginners will learn from my VR workflow and the seasoned researchers will learn from my techniques. In this blogpost, I'm discussing a bug I found in nf_tables in the Linux
前回，マスター方程式を導きましたが，マスター方程式は時間微分と状態の積分からなる方程式なので大きな自由度の系を扱うにはなかなか複雑なものでした．今回は，状態の間の「近さ」に着目することで，マスター方程式を形式的に微分方程式の形にすることを目標にします．ただし，状態間の距離を考えることに意味がないような状況もあるので，位置や密度など，定量的に状態間の「距離」を測れるようなものに限定して考えることになります． クラマース・モヤル方程式前回導いたマスター方程式とは，過程がマルコフ過程であるときに，ある時刻においてある状態を取るような確率密度の時間発展を記述する方程式のことで $$ \newcommand{\pd}[2]{\frac{\partial #1 }{\partial #2 }} \pd{}{t} f(x,t) = \int dx' \Bigl[ -W(x' \mid x,t) f(x,
LambdaLisp - A Lisp Interpreter That Runs on Lambda Calculus
LambdaLisp is a Lisp interpreter written as an untyped lambda calculus term. The input and output text is encoded into closed lambda terms using the Mogensen-Scott encoding, so the entire computation process solely consists of the beta-reduction of lambda calculus terms. When run on a lambda calculus interpreter that runs on the terminal, it presents a REPL where you can interactively define and e
Temporal Python SDK: A Durable, Distributed Asyncio Event Loop We are excited about the GA release of the Temporal Python SDK. Python is now a fully-supported workflow language in Temporal, and our use of native asyncio constructs makes it a perfect fit for Python developers looking to write durable workflows. Here are some links to read more about Temporal Python: Repository Application Developme
Implement tcl in tcl
Maybe someone has already done this. But the question in my mind is how much of Tcl can be implimented in Tcl itself. Obviously you can't implement system calls in Tcl, but you could implement just about everthing else. What commands/parts of Tcl would be in the minimal set? Earl Johnson Minimal set"set" both scalar and array modes."eval" command"unknown" "string index" command"string length" comm
Emotet vs EmoCheck
Emotet vs EmoCheck Emotet開発者との戦い NTT 社会情報研究所 谷 知亮 JPCERT/CC インシデントレスポンスグループ 喜野 孝太 JPCERT/CC インシデントレスポンスグループ 佐條 研 $whoami Tomoaki Tani Kota Kino Ken Sajo NTT 社会情報研究所社会理論 研究プロジェクトに所属。主にマ ルウェア分析技術や脆弱性分析 技術の研究を行っている。以前 はJPCERT/CCでマルウェア分 析やインシデント対応に従事して いた。Virus Bulletin, CODEBLUE, BotConf, BsidesLV, BlackHat USA Arsenalなどで講演。 国内ITベンダーでのメールセキュ リティ製品の導入支援業務を経 て、2019年よりJPCERT/CC。 現 在は、標的型攻撃への対応も含 んだ、日本国内
The Watermelon Operator
The Watermelon Operator Sep 24, 2024 In these two most excellent articles, https://without.boats/blog/let-futures-be-futures and https://without.boats/blog/futures-unordered, withoutboats introduces the concepts of “multi-task” and “intra-task” concurrency. I want to revisit this distinction — while I agree that there are different classes of patterns of concurrency here, I am not quite satisfied
"Get or create" is a very common operation for syncing data in the database, but implementing it correctly may be trickier than you may expect. If you ever had to implement it in a real system with real-life load, you may have overlooked potential race conditions, concurrency issues and even bloat! In this article I explore ways to "get ot create" in PostgresSQL. Illustration by Abstrakt Design Ta
How to improve RAG peformance ? — Advanced RAG Patterns — Part2
In the realm of experimental Large Language Models (LLMs), creating a captivating LLM Minimum Viable Product (MVP) is relatively straightforward, but achieving production-level performance can be a formidable task, especially when it comes to building a high-performing Retrieval-Augmented Generation (RAG) pipeline for in-context learning. This post, part of the “Advanced RAG Patterns” series, delv
An AI-native software engineer is one who deeply integrates AI into their daily workflow, treating it as a partner to amplify their abilities. This requires a fundamental mindset shift. Instead of thinking “AI might replace me” an AI-native engineer asks for every task: “Could AI help me do this faster, better, or differently?”. The mindset is optimistic and proactive - you see AI as a multiplier
Fitting a Forth in 512 bytes
Fitting a Forth in 512 bytes June 10, 2021 · 31 minute read This article is part of the Bootstrapping series, in which I start from a 512-byte seed and try to bootstrap a practical system. Software is full of circular dependencies if you look deep enough. Compilers written in the language they compile are the most obvious example, but not the only one. To compile a kernel, you need a running kerne
GitHub - ComfyUI-Workflow/awesome-comfyui: A collection of awesome custom nodes for ComfyUI
ComfyUI-Gemini_Flash_2.0_Exp (⭐+172): A ComfyUI custom node that integrates Google's Gemini Flash 2.0 Experimental model, enabling multimodal analysis of text, images, video frames, and audio directly within ComfyUI workflows. ComfyUI-ACE_Plus (⭐+115): Custom nodes for various visual generation and editing tasks using ACE_Plus FFT Model. ComfyUI-Manager (⭐+113): ComfyUI-Manager itself is also a cu
Django for Startup Founders: A better software architecture for SaaS startups and consumer apps
In an ideal world, startups would be easy. We'd run our idea by some potential customers, build the product, and then immediately ride that sweet exponential growth curve off into early retirement. Of course it doesn't actually work like that. Not even a little. In real life, even startups that go on to become billion-dollar companies typically go through phases like: Having little or no growth fo
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
The Prompt Engineering Playbook for Programmers
GPT in 60 Lines of NumPy | Jay Mody
Changing std::sort at Google’s Scale and Beyond
kyju.org - Piccolo - A Stackless Lua Interpreter
February 2021 (version 1.54)
Patterns for Building LLM-based Systems & Products
0.8.0 Release Notes ⚡ The Zig Programming Language
Advent of Code on the Nintendo DS
So You Want To Remove The GVL?
January 2024 (version 1.86)
0.10.0 Release Notes ⚡ The Zig Programming Language
Engineering for Slow Internet – brr
Who needs Graphviz when you can build it yourself?
What's New in Emacs 28.1?
Vim9 script for Python Developers · GitHub
Rust, reflection and access rules
Scientific Computing in Rust - aftix's dominion
cuneicode, and the Future of Text in C
PROJEKT: OVERFLOW
【物理数学】フォッカー・プランク方程式【確率論③】｜kT@物理・化学
Temporal Python SDK | Durable Asyncio Event Loop
How to Get or Create in PostgreSQL
The AI-Native Software Engineer