Processing large JSON files in Python without running out of memory

Processing large JSON files in Python without running out of memory by Itamar Turner-Trauring Last updated 06 Jan 2023, originally created 14 Mar 2022 If you need to process a large JSON file in Python, it’s very easy to run out of memory. Even if the raw data fits in memory, the Python representation can increase memory usage even more. And that means either slow processing, as your program swaps

[misshiki]

“メモリを使い果たすことなくPythonで大きなJSONファイルを処理する”

[noritada]

どんなデータでもよくある話ではあるが、streamingすることでメモリサイズを減らすのが1つのアプローチ。