Authored byShirshanka Das Co-founder and CTO @ Acryl | Founder DataHub project | Ex-LinkedIn December 7, 2020 When I started my journey at LinkedIn ten years ago, the company was just beginning to experience extreme growth in the volume, variety, and velocity of our data. Over the next few years, my colleagues and I in LinkedIn’s data infrastructure team built out foundational technology like Espr
Facebookから5億人を超えるユーザーの個人情報が流出したことが報じられていますが、新たに、ビジネス特化型SNS「LinkedIn」からも5億人分の個人情報が流出し、ハッキングフォーラムで取引されていることが判明しました。 Scraped data of 500 million LinkedIn users being sold online, 2 million records leaked as proof | CyberNews https://cybernews.com/news/stolen-data-of-500-million-linkedin-users-being-sold-online-2-million-leaked-as-proof-2/ LinkedInからの個人情報の流出は、セキュリティ関連メディアのCyberNewsがハッキングフォーラム上で「5億人分のL
Open Source Open sourcing Kube2Hadoop: Secure access to HDFS from Kubernetes Co-authors: Cong Gu, Abin Shahab, Chen Qiang, and Keqiu Hu Editor's note: This blog has been updated. LinkedIn AI has been traditionally Hadoop/YARN based, and we operate one of the world’s largest Hadoop data lakes, with over 4,500 users and 500PB of data. In the last few years, Kubernetes has also become very popular at
ThirdEye: LinkedIn’s Business-wide Monitoring Platform 26th Sept 2019 Akshay Rai Senior Software Engineer Strata Data Conference, NY Agenda 1 What is ThirdEye? 2 ThirdEye @ LinkedIn 3 Anomaly Detection & Analysis 4 Overview of Entity Monitoring Root-Cause Analysis Anomaly Detection Business Impact Time Incident Detection Analysis Recovery ← MTTD (Mean time to detect) ← MTTR (Mean time to restore)
Authored byMars Lan Co-Founder & CTO at Metaphor | Co-creator of DataHub August 14, 2019 Co-authors: Mars Lan, Seyi Adebajo, Shirshanka Das Editor’s note: Since publishing this blog post, the team open sourced DataHub in February 2020. You can read more on the journey of open sourcing the platform here. As the operator of the world’s largest professional network and the Economic Graph, LinkedIn’s
Open Source Open sourcing DataHub: LinkedIn’s metadata search and discovery platform Co-authors: Kerem Sahin, Mars Lan, and Shirshanka Das Finding the right data quickly is critical for any company that relies on big data insights to make data-driven decisions. Not only does this impact the productivity of data users (including analysts, machine learning developers, data scientists, and data engin
How LinkedIn, Uber, Lyft, Airbnb and Netflix are Solving Data Management and Discovery for Machine Learning Solutions When comes to machine learning, data is certainly the new oil. The processes for managing the lifecycle of datasets are some of the most challenging elements of large scale machine learning solutions. Data ingestion, indexing, search, annotation, discovery are some of the aspects r
In modern data-driven businesses, the complexity that arises from fast-paced analytics, data mining and ETL processes makes metadata increasingly important. In this blog post, we share our own journey and a new open source effort that aims to boost productivity and data provenance. WhereHows, a project of the LinkedIn Data team, works by creating a central repository and portal for the processes,
Authored byYen-Jung Chang ML Research Scientist at Facebook February 20, 2020 Co-authors: Yen-Jung Chang, Yang Yang, Xiaohui Sun, and Tie Wang At LinkedIn, ThirdEye is the backbone of our monitoring toolkit. We use it to keep track of a variety of metrics, whether it be related to production infrastructure and AI model performance, or business impact, such as page view or click count. It’s a key q
dl.acm.org LinkedinでInvalidなA/B テスト検知するための方法論を紹介している論文を読みました A/Bテストにおけるメトリック解析の話から、丁寧にメトリックを分解し、検知ロジックを提案しています。LinkedinのA/B テストプラットフォームで実際に動いてるみたいです Introduction まずはじめに、invalidなA/BテストにはInternally または Externally なテストがあることを紹介しています Internally invalid とは? Treatment と Control間の差がTreatmentによる効果と結論づけれるのが本来のA/B テストの魅力の1つですが、その差がTreatmentによる効果ではない ことを言います 特にこの論文では、incomparable samples(つまり、比較不可なサンプル群)における I
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く