By Ammar Khaku IntroductionIn a microservice architecture such as Netflix’s, propagating datasets from a single source to multiple downstream destinations can be challenging. These datasets can represent anything from service configuration to the results of a batch job, are often needed in-memory to optimize access and must be updated as they change over time. One example displaying the need for d
With the increasing number of individuals performing data science, in every organization and team, there is a proliferation of dataset versions at various stages of data analysis. More often than not, these dataset versions are stored in an ad-hoc manner in shared file systems, leading to massive redundancy and duplication, and making it impossible to keep track of and find specific versions. Orph
import cv2 import numpy as np from matplotlib import pyplot as plt import os # edges.py reads an image and outputs transformed image def make_edges(image): img = cv2.imread(image) tail = os.path.split(image)[1] edges = cv2.Canny(img,100,200) plt.imsave(os.path.join("/pfs/out", os.path.splitext(tail)[0]+'.png?as=webp'), edges, cmap = 'gray') # walk images directory and call make_edges on every file
Microservices and the art of taming the Dependency Hell Monster Michael Bryzek Cofounder & ex-CTO Gilt @mbryzek mbryzek@alum.mit.edu Dependency Hell • What is it and how does it happen? • How do we mitigate? • API design must be First Class • Backward and Forward Compatibility • Accurate Documentation • Generated client libraries http://en.wikipedia.org/wiki/Dependency_hell Dependency hell is a c
DataHub A Data Ecosystem for Individuals, Teams and People What is DataHub? For End Users: A way to store your data centrally, without having to set up your own database A way to collaborate with others A way to seamlessly share your data with friends and colleagues A suite of tools to process your data For Developers: A database-agnostic, language-agnostic, MITREid (OpenID + OAuth2) integrated pl
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く