msyktのブックマーク - はてなブックマーク

The columnar roadmap: Apache Parquet and Apache Arrow - DataWorks Summit DataWorks Summit
The Hadoop ecosystem has standardized on columnar formats—Apache Parquet for on-disk storage and Apache Arrow for in-memory. With this trend, deep integration with columnar formats is a key differentiator for big data techno logies. Vertical integration from storage to execution greatly improves the latency of accessing data by pushing projections and filters to the storage layer, reducing time spe
msykt 2018/06/21
parquet

arrow
リンク
Using LLVM to accelerate processing of data in Apache Arrow - DataWorks Summit DataWorks Summit
Most query engines follow an interpreter-based approach where a SQL query is translated into a tree of relational algebra operations then fed through a conventional tuple-based iterator model to execute the query. We will explore the overhead associated with this approach and how the performance of query execution on columnar data can be improved using run-time code generation via LLVM. Generally
msykt 2018/06/16
リンク
1

はてなブックマーク