PEP 703 - 兼雑記
https://peps.python.org/pep-0703/ Python の GIL 外す話。これすごく楽しい読みものでした。参照カウントのところが一番人気だと思うのですが、他のところも色々良い。こういう、「んーこういうことするとこういう問題が起きない？」と思ったら次の章くらいでそれが説明される、みたいな読みものは大変好きです 参照カウント: オブジェクトっていうのは作ったスレッドが解放するというのがほとんどなんだから、その場合はロックをいらなくする、他に渡ったら普通の参照カウントぽくする、という話。 Swift に 2018 年に導入された 話らしい。他のスレッドに渡された後で DECREF すると他スレッド用の参照カウントが負になりうるのだけど、その時に queue に入れるということをして、ややこしいので、なんかこれ無しですむ方法はないのかなぁ……と Immortalize
Handling Concurrency Without Locks
Concurrency is not very intuitive. You need to train your brain to consider what happens when multiple processes execute a certain code block at the same time. There are several issues I often encounter: Failing to recognize potential concurrency issues: It's not uncommon for both beginner and seasoned developers to completely miss a potential concurrency problem. When this happens, and the concur
【Python】asyncioの代替ライブラリtrioを調べてみた - 歩いたら休め
この記事はPython その2 Advent Calendar 2018の遅れてきた15日目の記事です。 最近、クローリング用のプログラムのasyncioを使った並行処理のプログラミングをしており、「asyncioのベストプラクティス」という趣旨の記事を書こうと思っていたのですが、自分自身感覚を掴めておらず、「あれ？これってasyncioのAPI自体も微妙なのでは？」「そもそもasyncioのドキュメントのドキュメントを読んでも、どの機能をどう組み合わせて使えばいいか分からない」と思ってしまっていて記事を書きかねていました。 ただ、知人から「trioというライブラリのAPI設計が素晴らしい」と薦めて頂いたのと、あまりにも待たせてしまうのも申し訳ないのでひとまず「調べてみた」レベルでアウトプットを出します。 私はまだ並列/並行/非同期処理に詳しいわけではないので、大きな思い違いをしている可能
Decorated Concurrency - Python multiprocessing made really really easy - Peterbe.com
Decorated Concurrency - Python multiprocessing made really really easy tl;dr There's a new interesting wrapper on Python multiprocessing called deco, written by Alex Sherman and Peter Den Hartog, both at University of Wisconsin - Madison. It makes Python multiprocessing really really easy. The paper is here (PDF) and the code is here: https://github.com/alex-sherman/deco. This library is based on
Concurrent Crawling with Go and Python – Venkata Mahalingam
June 30, 2014 What crawls better, gopher or a snake? :) I was recently talking to my friend who is also a fellow python programmer, about the system he built to crawl millions of RSS feeds very fast. He has his solution working based on Gevent and I was advocating him to take a look at Go, after being impressed with it after hearing Brad Fitzpatrick's talk at the GoSF meetup on how they sped up dl
Parallelism and Concurrency with Python
Presented May 2014. For more information on PyParallel, see: https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores …
Python - paralellizing CPU-bound tasks with concurrent.futures - Eli Bendersky's website
A year ago, I wrote a series of posts about using the Python multiprocessing module. One of the posts contrasted compute-intensive task parallelization using threads vs. processes. Today I want to revisit that topic, this time employing the concurrent.futures module which is part of the standard library since Python 3.2 First of all, what are "futures"? The Wikipedia page says: In computer science
Python Concurrency: Porting from a Queue to Multiprocessing - Mouse Vs Python
Earlier this week, I wrote a simple post about Python’s Queues and demonstrated how they can be used with a threading pool to download a set of PDFs from the United States Internal Revenue Service’s website. Today I decided to try “porting” that code over to Python’s multiprocessing module. As one of my readers pointed out, Python’s Queues and threads are limited to running on only one core due to
Python Concurrency: An Example of a Queue - Mouse Vs Python
Python comes with a lot of cool concurrency tools builtin, such as threads, Queues, semaphores and multiprocessing. In this article, we’ll spend some time learning how to use Queues. A Queue can be used for first-in-first out or last-in-last-out stack-like implementations if you just use them directly. If you’d like to see that in action, see the Hellman article at the end of this post. We’re goin
asyncoro: Asynchronous, Concurrent, Distributed Programming with Python — asyncoro 4.5.3 documentation
» asyncoro: Asynchronous, Concurrent, Distributed Programming with Python asyncoro: Asynchronous, Concurrent, Distributed Programming with Python¶ Note asyncoro has been renamed pycos to better reflect its functionality and to avoid confusion with asyncore module. asyncoro is a Python framework for asynchronous, concurrent, network, distributed programming and distributed computing, using generato
:A Curious Course on Coroutines and Concurrency
Copyright (C) 2009, All Rights Reserved David Beazley http://www.dabeaz.com Presented at PyCon 2009, March 25, 2009. Related Tutorials Generator Tricks for Systems Programmers. Presented at PyCon 2008 (Chicago). Generators: The Final Frontier. Presented at PyCon 2014 (Montreal). Introduction This tutorial is a practical exploration of using Python coroutines (extended generators) for solving probl
Futures: Concurrency in Python 3, Part 1
02 Jun 2012 This series of articles is on writing concurrent software in Python 3 using the concurrent.futures module. In this article I describe the problem that I'm trying to solve and talk about the factors which lead me to a threaded, concurrent design. If you're not interested in background and reasoning, skip down to the last section of this article. For readers who may not be familar with t
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Real Multithreading is Coming to Python - Learn How You Can Use It Now | Martin Heinz | Personal Website & Blog
GitHub - alex-sherman/deco
Wasp's Nest: The Read-Copy-Update Pattern In Python