clavierのブックマーク - はてなブックマーク

LyftLearn: ML Model Training Infrastructure built on Kubernetes
Authors: Vinay Kakade, Shiraz Zaman IntroductionIn a previous blog post, we discussed the architecture of Feature Service, which manages Machine Learning (ML) feature storage and access at Lyft. In this post, we’ll discuss the architecture of LyftLearn, a system built on Kubernetes, which manages ML model training as well as batch predictions. ML forms the backbone of the Lyft app and is used in d
clavier 2021/05/18
model

kubernetes

infrastructure

image

machinelearning

mlops

lyft
リンク
Open Sourcing Amundsen: A Data Discovery And Metadata Platform
By Tao Feng, Jin Hyuk Chang, Tamika Tannis, Daniel Won In a modern data-driven company like Lyft, every interaction on the platform is powered by data. The challenges that arise from complex data generation, ETL processes, and analytics make metadata significantly important. Moreover, the types of data resources are constantly increasing. At Lyft, these resources include SQL tables and views in Re
clavier 2019/12/01
data

metadata

architecture

データ分析

search
リンク
Presto Infrastructure at Lyft
OverviewEarly in 2017 we started exploring Presto for OLAP use cases and we realized the potential of this amazing query engine. It started as an adhoc querying tool for data engineers and analysts to run SQL in a faster way to prototype their queries, when compared to Apache Hive. A lot of internal dashboards were powered by AWS-Redshift back then and it had data storage and compute coupled toget
clavier 2019/09/13
presto
リンク
1