WSDM 2019 : How A/B Tests Could Go Wrong : Automatic Diagnosis of Invalid Online Experiments を読んだ - OKIYUKI99 Blog
dl.acm.org LinkedinでInvalidなA/B テスト検知するための方法論を紹介している論文を読みました A/Bテストにおけるメトリック解析の話から、丁寧にメトリックを分解し、検知ロジックを提案しています。LinkedinのA/B テストプラットフォームで実際に動いてるみたいです Introduction まずはじめに、invalidなA/BテストにはInternally または Externally なテストがあることを紹介しています Internally invalid とは？ Treatment と Control間の差がTreatmentによる効果と結論づけれるのが本来のA/B テストの魅力の1つですが、その差がTreatmentによる効果ではない ことを言います 特にこの論文では、incomparable samples（つまり、比較不可なサンプル群）における I
Introduction: Technical Paper on LinkedIn's A/B Testing Platform
Introduction: Technical Paper on LinkedIn's A/B Testing Platform XLNT is the end-to-end A/B testing platform used at LinkedIn to not only solve the day-to-day A/B testing needs across the company, but also sophisticated use cases that are prevalent in a social network setting. With all the lessons learned from using the platform, we decided to take an in-depth look at how we approach A/B testing a
XLNT Platform: Driving A/B Testing at LinkedIn
A/B testing is an indispensable driver behind LinkedIn’s data culture. We run more than 200 experiments in parallel daily and that number is growing quickly. We realized early on that ad hoc testing would only take us so far. We needed to change our approach to allow us to quickly quantify the impact of any A/B test in a scientific and controlled manner across LinkedIn.com and our apps. All of the
Real-time Analytics at Massive Scale with Pinot
Two years ago we hit a wall. The scale of LinkedIn’s data was growing beyond what we could analyze. At the same time, our members needed their analytics and insights in real-time. We needed a solid solution that would grow with LinkedIn and serve as the platform to power all of our analytics needs across the company at web-scale. In this post, we will share how and why we built a distributed real-
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Email Volume Optimization at LinkedIn
Evaluating Mobile App Release
Network A/B Testing: From Sampling to Estimation | LinkedIn Engineering
From Infrastructure to Culture: A/B Testing Challenges in Large Scale Social Networks
talk-ya
