columnarの検索結果1 - 3 件 / 3件

• あとで読む

  GitHub - lancedb/lance: Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more i

  • 6 users
  • github.com/lancedb
  • テクノロジー
  • 2023/02/12

  Lance is a modern columnar data format that is optimized for ML workflows and datasets. Lance is perfect for: Building search engines and feature stores. Large-scale ML training requiring high performance IO and shuffles. Storing, querying, and inspecting deeply nested data for robotics or large blobs like images, point clouds, and more. The key features of Lance include: High-performance random a

  • データ
  • あとで読む

  
• あとで読む

  GitHub - kelindar/column: High-performance, columnar, in-memory store with bitmap indexing in Go

  • 4 users
  • github.com/kelindar
  • テクノロジー
  • 2021/06/21

  Optimized, cache-friendly columnar data layout that minimizes cache-misses. Optimized for zero heap allocation during querying (see benchmarks below). Optimized batch updates/deletes, an update during a transaction takes around 12ns. Support for SIMD-enabled aggregate functions such as "sum", "avg", "min" and "max". Support for SIMD-enabled filtering (i.e. "where" clause) by leveraging bitmap inde

  • database

  
• あとで読む

  Guide to File Formats for Machine Learning: Columnar, Training, Inferencing, and the Feature Store

  • 3 users
  • towardsdatascience.com
  • テクノロジー
  • 2019/11/09

  TLDR; Most machine learning models are trained using data from files. This post is a guide to the popular file formats used in open source frameworks for machine learning in Python, including TensorFlow/Keras, PyTorch, Scikit-Learn, and PySpark. We will also describe how a Feature Store can make the Data Scientist’s life easier by generating training/test data in a file format of choice on a file

  • data