Parquet: more than just "Turbo CSV"

Parquet is an efficient, binary file format for table data. Compared to csv, it is: Quicker to read Quicker to write Smaller On a real world 10 million row financial data table I just tested with pandas I found that Parquet is about 7.5 times quicker to read than csv, ~10 times quicker to write and a about a fifth of the size on disk. So way to think of Parquet is as "turbo csv" - like csv, just f

[tmatsuu]

CSVよりも断然良いParquet形式のすゝめ。CSVより読み書き高速でサイズが小さく、型があり、エンコーディング定義があり、nullもある。欠点はデータ構造上ストリーム処理ができない。なるほど。