Unlocking the Power of JunoDB: PayPal’s Key-Value Store Goes Open-Source
Today we are delighted to share JunoDB as an open-source project on Github, allowing others to benefit from our efforts to have an extremely scalable, secure and highly available NoSQL infrastructure. JunoDB is a distributed key-value store that plays a critical role in powering PayPal’s diverse range of applications. Virtually every core back-end service at PayPal relies on JunoDB, from login to
The race to reconnect Tonga
The race to reconnect Tonga How engineers will repair the undersea communications cable severed by the recent volcanic eruption A global map of undersea communication cables stretching across the Pacific Ocean, with China to the west and the U.S. to the east. The map shows the Tonga volcanic eruption and several cables running near the island nation. The South Pacific nation of Tonga was all but c
Testing sync at Dropbox
…and how we rewrote the heart of sync with confidence. Executing a full rewrite of the Dropbox sync engine was pretty daunting. (Read more about our goals and how we made the decision in our previous post here.) Doing so meant taking the engine that powers Dropbox on hundreds of millions of user’s machines and swapping it out mid-flight. To pull this off, we knew we would need a serious investment
Scaling Datastores at Slack with Vitess - Slack Engineering
Rafael Chacón Staff Software Engineer, Infrastructure From the very beginning of Slack, MySQL was used as the storage engine for all our data. Slack operated MySQL servers in an active-active configuration. This is the story of how we changed our data storage architecture from the active-active clusters over to Vitess — a horizontal scaling system for MySQL. Vitess is the present and future of Dat
Why Zoom Chose Oracle Cloud Over AWS and Maybe You Should Too
04.28.2020 Why Zoom Chose Oracle Cloud Over AWS and Maybe You Should Too By Corey Quinn Updated: May 1, 2020. See the bottom of this article for some clarifications. Today, news broke that Zoom signed a deal with Oracle Cloud to host their cloud infrastructure, beating out AWS,… Updated: May 1, 2020. See the bottom of this article for some clarifications. Today, news broke that Zoom signed a deal
AWS News Blog New – Announcing Amazon AppFlow Software as a service (SaaS) applications are becoming increasingly important to our customers, and adoption is growing rapidly. While there are many benefits to this way of consuming software, one challenge is that data is now living in lots of different places. To get meaningful insights from this data, we need to have a way to analyze it, and that c
The history and future roadmap of the AWS CloudFormation Registry | Amazon Web Services
AWS DevOps Blog The history and future roadmap of the AWS CloudFormation Registry AWS CloudFormation is an Infrastructure as Code (IaC) service that allows you to model your cloud resources in template files that can be authored or generated in a variety of languages. You can manage stacks that deploy those resources via the AWS Management Console, the AWS Command Line Interface (AWS CLI), or the
Open Sourcing the Netflix Domain Graph Service Framework: GraphQL for Spring Boot
Netflix has developed a Domain Graph Service (DGS) framework and it is now open source. The DGS framework simplifies the implementation of GraphQL, both for standalone and federated GraphQL services. Our framework is battle-hardened by our use at scale. By open-sourcing the project, we hope to contribute to the Java and GraphQL communities and learn from and collaborate with everyone who will be u
Rebuilding Twitter’s public API
Today we launched the new Twitter API v2. Our first launch of a public API was in 2006 and shortly after, we began building API access to new features with the intention of opening our platform and inviting developers to build the future with us. Six years after the first launch, in 2012, we released the v1.1 API that introduced new requirements and stricter policies needed to curb abuse and prote
In Liu Cixin’s work, civilizations engage in a Hobbesian struggle for survival.Illustration by Robert Beatty Two rival civilizations are battling for supremacy. Civilization A is stronger than Civilization B and is perceived by Civilization B as a grave threat; its position, however, is more fragile than it seems. Neither side hesitates to employ espionage, subterfuge, and surveillance, because th
The Citadel Architecture at AppSignal | AppSignal Blog
DHH just coined the term "Citadel," which finally gives us an excellent way to reference how we approach tech at AppSignal. We said, "Hey, this is us! Our thing has a name now". In addition to the Majestic Monolith, someone should write up the pattern of The Citadel: A single Majestic Monolith captures the majority mass of the app, with a few auxiliary outpost apps for highly specialized and diver
Introducing the next generation of Cloud Functions | Google Cloud Blog
Today, we are introducing Cloud Functions (2nd gen) into public preview. This next-generation of our Functions-as-a-Service product comes with an advanced feature set giving you more powerful infrastructure, advanced control over performance and scalability, more control around the functions runtime, and triggers from over 90 event sources. Further, the infrastructure is powered by Google Cloud’s
Preparing for the Systems Design and Coding Interview
At Big Tech and high-growth startups, coding and systems design interviews are common - and fairly standard. A lot of people have asked me for preparation advice for these. Here is what I used when getting ready for an E5/E6 Facebook interview, and the one at Uber - where I was hired as a senior software engineer (L5). It's the same resources I recommend to people who are preparing for Big Tech or
Disaster Recovery (DR) Architecture on AWS, Part III: Pilot Light and Warm Standby | Amazon Web Services
AWS Architecture Blog Disaster Recovery (DR) Architecture on AWS, Part III: Pilot Light and Warm Standby In this blog post, you will learn about two more active/passive strategies that enable your workload to recover from disaster events such as natural disasters, technical failures, or human actions. Previously, I introduced you to four strategies for disaster recovery (DR) on AWS. Then we explor
The Workers Browser Rendering API allows developers to programmatically control and interact with a headless browser instance and create automation flows for their applications and products. Since the private beta announcement, based on the feedback we've been receiving and our own roadmap, the team has been working on the developer experience and improving the platform architecture for the best p
Skip the API, Ship Your Database Author Name Ben Johnson @benbjohnson @benbjohnson Image by Annie Ruygt With Fly.io, you can get your app running globally in a matter of minutes, and with LiteFS, you can run SQLite alongside your app! Now we’re introducing LiteFS Cloud: managed backups and point-in-time restores for LiteFS. Try it out for yourself! My favorite part about building tools is discover
npm security update: Attack campaign using stolen OAuth tokens
Securitynpm security update: Attack campaign using stolen OAuth tokensnpm's impact analysis of the attack campaign using stolen OAuth tokens and additional findings. As of June 2, 2022, GitHub has completed directly notifying all impacted users for whom we were able to detect abuse from the attack on npm. If you have not received a notification directly from GitHub, we do not have evidence that yo
At LocalStack we are committed to constantly improve the cloud dev experience. Here is how our AWS Server Framework and a new snapshot testing framework help us to stay on top of AWS changes. Parity for LocalStack means that when you, as a cloud developer, make an AWS API call to LocalStack’s cloud emulator, it behaves the same way AWS would. Keeping parity with AWS has been our mission at LocalSt
ネットワークなんて触ったことないから分からない――クラウド世代のための「IPネットワーク」超入門：AWSで学ぶクラウド時代のネットワーク基礎知識（1） これまであまり物理的なネットワークに触れてこなかったエンジニアを対象に、AWSを用いてネットワークの基礎知識を解説する連載。初回は、基本的なIPネットワークの概念を解説し、Amazon VPCの操作手順を示す。 なぜ、今「IPネットワーク」を学ぶべきなのか クラウドや無線LANが当たり前になった昨今、以前はLANケーブルやアプライアンス、サーバなどで物理的に理解することができた「IPネットワーク」について、意識することが難しくなっています。一方で、IaaS（Infrastructure as a Service）やPaaS（Platform as a Service）などクラウドを使いこなす上では、ネットワークについて基本から分かっている必
servicemesh.es
What is a Service Mesh? A service mesh is a dedicated infrastructure layer that adds features to a network between services. It allows to control traffic and gain insights throughout the system. Observability, traffic shifting (for canary releasing), resiliency features (such as circuit breaking and retry/timeout) and automatic mutual TLS can be configured once and enforced in a decentralized fash
Not Using Serverless Yet? Why You Need to Care about re:Invent 2019’s Serverless Launches
Why Container Devs Need to Care about re:Invent 2019’s Serverless Launches Another re:Invent is over, and an army of weary AWS employees have flown back home, their launch tasks complete. The AWS Serverless teams, including AWS Lambda, Amazon API Gateway, AWS Step Functions, Amazon EventBridge and others were especially prolific in in the run up to, and during, re:Invent. But what does it all mean
In this presentation, Joe Williams describes the architecture of the GitHub Load Balancer (GLB). GitHub built a resilient custom solution on top of HAProxy to intelligently route requests coming from a variety of different clients including Git, SSH and MySQL. The GLB is split into two major components: the GLB Director and GLB proxies. The latter is built upon HAProxy, which provides many benefit
New – Port Forwarding Using AWS System Manager Session Manager | Amazon Web Services
AWS News Blog New – Port Forwarding Using AWS System Manager Session Manager I increasingly see customers adopting the immutable infrastructure architecture pattern: they rebuild and redeploy an entire infrastructure for each update. They very rarely connect to servers over SSH or RDP to update configuration or to deploy software updates. However, when migrating existing applications to the cloud,
Store and Access Time Series Data at Any Scale with Amazon Timestream – Now Generally Available | Amazon Web Services
AWS News Blog Store and Access Time Series Data at Any Scale with Amazon Timestream – Now Generally Available Time series are a very common data format that describes how things change over time. Some of the most common sources are industrial machines and IoT devices, IT infrastructure stacks (such as hardware, software, and networking components), and applications that share their results over ti
“ゼロトラスト”は死んだ？ ノークリサーチ岩上氏に聞く「セキュリティの現在地と現実解」：輝ける情シスになるためには何が必要か～求められる人材に変化しよう～（3） 最近、「ゼロトラスト」という語をあまり聞かなくなったと感じる読者もいるだろう。ノークリサーチ岩上氏にその理由と背景を聞いたところ、システムへの侵入という最悪の事態を防ぐために今すべきことが明らかになった。 コロナ禍による非対面化の進行とSaaS（Software as a Service）の普及の影響で、企業内外のネットワークから重要データへのアクセスが増加した。その結果、企業の“内”と“外”で安全を定義するのではなく、基本的に誰も信用せず厳密な認証・認可によってシステムへのアクセスをコントロールする“ゼロトラストアーキテクチャ”構築の重要性が叫ばれて久しい。 しかし、コロナ禍真っただ中でテレワークシフトが盛んだった頃と比較すると
Helmの勉強のために、各サブコマンドで何ができるか調べました。 環境情報 EKS Workshopで使う、Cloud 9インスタンス上で検証しています。Linuxです。 Helmのバージョンは以下です。 $ helm version version.BuildInfo{Version:"v3.5.0", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"clean", GoVersion:"go1.15.6"} completion generate autocompletions script for the specified shell シェル自動補完のためのスクリプトを吐いてくれます。 各シェル用のサブコマンドがあります。 Available Commands: bash generate au
How we handle 80TB and 5M page views a month for under $400
Irregular long-form blog. For the latest news, see the posts on Patreon. Note: All stats and figures below are true only at the time of writing, and will certainly change over time. Running a massively popular website and asset resource while being funded primarily by donations has always been a core challenge of Poly Haven. It’s easy to throw a bunch of files on Amazon S3 and call it a day, but y
Blog - Private Cloud Compute: A new frontier for AI privacy in the cloud - Apple Security Research
Private Cloud Compute: A new frontier for AI privacy in the cloud Written by Apple Security Engineering and Architecture (SEAR), User Privacy, Core Operating Systems (Core OS), Services Engineering (ASE), and Machine Learning and AI (AIML) Apple Intelligence is the personal intelligence system that brings powerful generative models to iPhone, iPad, and Mac. For advanced features that need to reaso
DevOps Topologies : DevOps 文化を築くベストプラクティスな組織構造とは - kakakakakku blog
DevOps 文化を築くときに知っておくと便利なサイト「DevOps Topologies」を読んだ．「アンチパターン = DevOps Anti-Types」な組織構造（計7種類）と「ベストプラクティス = DevOps Team Topologies」な組織構造（計9種類）が紹介されている．サイトではそれぞれの組織構造の解説と関係性を図解した絵が載っていてわかりやすい！本記事ではサイトを翻訳するのではなく，あくまで個人的なメモとしてまとめる．なお，日本語訳も参考程度に載せておいた！ web.devopstopologies.com 目次 1. DevOps Anti-Types Anti-Type A: Dev and Ops Silos（Dev と Ops のサイロ化） Anti-Type B: DevOps Team Silo（DevOps チームのサイロ化） Anti-Type
I’m excited to announce that Gatsby Inc. is joining Netlify in order to spread the growth of Gatsby and composable architectures across the entire web. To see how we got here, let’s start by rewinding back in time. Eight years ago, I was a front-end developer, and I fell in love with React. At the time, React was a tiny emerging framework for apps, but I wanted to build a website with it. So I too
You've used Google Maps, but what about My Maps? This tool can come in handy for a variety of reasons. You've probably used Google Maps to find routes to addresses, search for stores and services, or to "explore" new places before you get there. My Maps is a service through Google Maps that lets you use the Google Maps infrastructure to attach information to locations and create custom maps. Here,
AWS Cloud Control API, a Uniform API to Access AWS & Third-Party Services | Amazon Web Services
AWS News Blog AWS Cloud Control API, a Uniform API to Access AWS & Third-Party Services Today, I am happy to announce the availability of AWS Cloud Control API a set of common application programming interfaces (APIs) that are designed to make it easy for developers to manage their AWS and third-party services. AWS delivers the broadest and deepest portfolio of cloud services. Builders leverage th
EngineeringProductFebruary service disruptions post-incident analysisIn-depth analysis of February service disruptions that impacted GitHub services. In late February, GitHub experienced multiple service interruptions that resulted in degraded service for a total of eight hours and 14 minutes over four distinct events. Unexpected variations in database load, coupled with an unintended configuratio
AWS Fargate Spot Now Generally Available | Amazon Web Services
AWS News Blog AWS Fargate Spot Now Generally Available Today at AWS re:Invent 2019 we announced AWS Fargate Spot. Fargate Spot is a new capability on AWS Fargate that can run interruption tolerant Amazon Elastic Container Service (Amazon ECS) Tasks at up to a 70% discount off the Fargate price. If you are familiar with EC2 Spot Instances, the concept is the same. We use spare capacity in the AWS c
Running Serverless Puppeteer with Workers and Durable Objects
Running Serverless Puppeteer with Workers and Durable Objects09/28/2023 Last year, we announced the Browser Rendering API – letting users running Puppeteer, a browser automation library, directly in Workers. Puppeteer is one of the most popular libraries used to interact with a headless browser instance to accomplish tasks like taking screenshots, generating PDFs, crawling web pages, and testing w
Preview: Amazon Security Lake – A Purpose-Built Customer-Owned Data Lake Service | Amazon Web Services
AWS News Blog Preview: Amazon Security Lake – A Purpose-Built Customer-Owned Data Lake Service To identify potential security threats and vulnerabilities, customers should enable logging across their various resources and centralize these logs for easy access and use within analytics tools. Some of these data sources include logs from on-premises infrastructure, firewalls, and endpoint security so
Under the hood: Amazon Elastic Container Service and AWS Fargate increase task launch rates | Amazon Web Services
Containers Under the hood: Amazon Elastic Container Service and AWS Fargate increase task launch rates Since 2015, hundreds of thousands of developers have chosen Amazon Elastic Container Service (Amazon ECS) as their orchestration service for cluster management. Developers trust Amazon ECS with the lifecycle of their mission-critical applications, from initial deployment to rolling out new versio
Tao of Node - Design, Architecture & Best PracticesMarch 14, 2022 • 48 minute read One of the main benefits of JavaScript is that it runs both in the browser and the server. As an engineer you need to master a single language and your skills will have a variety of applications. This is what drew me to Node in 2015 - I didn’t have to switch between languages and tech stacks. Node allows you to reus
Opinion | A Sports Event Shouldn’t Be a Superspreader. Cancel the Olympics. (Published 2021)
The Tokyo Olympics are in big trouble. Postponed by a year and slated to begin in July, the Olympics have become a political flash point in Japan, where almost 60 percent of the population opposes staging the Games this summer and where less than 2 percent of the population is vaccinated for Covid-19. The International Olympic Committee, local Olympic organizers and Japan’s ruling party maintain t
Zoom’s Encryption Is “Not Suited for Secrets” and Has Surprising Links to China, Researchers Discover
Meetings on Zoom, the increasingly popular video conferencing service, are encrypted using an algorithm with serious, well-known weaknesses, and sometimes using keys issued by servers in China, even when meeting participants are all in North America, according to researchers at the University of Toronto. The researchers also found that Zoom protects video and audio content using a home-grown encry
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
New – Announcing Amazon AppFlow | Amazon Web Services
Liu Cixin’s War of the Worlds
Workers Browser Rendering API enters open beta
Skip the API, Ship Your Database
LocalStack and AWS Parity Explained
ネットワークなんて触ったことないから分からない――クラウド世代のための「IPネットワーク」超入門
Inside the GitHub Load Balancer | HAProxyConf2019
“ゼロトラスト”は死んだ？ ノークリサーチ岩上氏に聞く「セキュリティの現在地と現実解」
Helmのサブコマンドを全部使ってみた | DevelopersIO
Gatsby is joining Netlify | Gatsby
5 Essential Google My Maps Features You Should Know
February service disruptions post-incident analysis
Tao of Node - Design, Architecture & Best Practices