こんにちは。プロダクト開発部の荒川 id:ad-sho-loko です。突然ですが、皆さんはこんな疑問を持ったことはありませんか？ データベースの内部実装はどうなっているのか？ トランザクションとはどのようなアルゴリズムで実現されているのか？ NoSQLが遅いのはなぜか？ 古典的なデータベースとは内部的にどのように違うの？ データベースを何かしらの形で利用しているのにも関わらず、意外と内部の仕組みを理解していない場合が多いかと思います。僕もそうです。*1 しかし、エンジニアたるもの、その仕組みを知ることは非常に重要です。僕もデータベースについて勉強しようといくつかの本やサイトを調べていたのですが、なかでもCMU（カーネギーメロン大学）のDatabase System Groupがアップロードしている講義が最も勉強になりました。 www.youtube.com そして本ブログでは、上記の講義
DALL-E3を実装日からほぼ毎日使用し面白さにとりつかれています。何となくで触っているため, いまいち思った画像が出来ないことも多々ありましたので今回まじめに作成法を勉強してみました。初級編と名前がついているのは高等テクニックを教えるほどの技術がないだけで後に上級編が控えているという意味ではないです。 0. はじめに DALL-E とはシンプルなテキストのみで画像がつくれるAIです。 簡単なテキストのみで画像生成語源は『ウォーリー探せ』と芸術家の『ダリ』から来ているみたいです。ウィーリーはある種の「探し物」をする, userが提示するテキストのプロンプトから隠された要素やまだ見ぬ画像を「探し出し」生成することらしいです。 ウォーリーをインスパイアした少年1. 問題点, 主に著作権やはり何と言っても著作権問題ではないでしょうか。現在法整備が進行しているところです。OpenAIはコンテンツポ
REST, GraphQL, and gRPC are 3 popular forms client-server and server-to-server communication. Choosing can be difficult, so this concise guide can help. In each section, an example will be provided to illustrate retrieving a user. REST Notes HTTP paths describing data, e.g. /users as a collection of users Easily discoverable data, e.g. user ID 3 would be at /users/3. All of the CRUD (Create Read U
Broken Ownership
Have you been in any of these situations? Managers make decisions that’s out of their leagues and everyone else in the team ends up paying for it. Knowledgeable people passively observe without bothering to contribute. Sometimes they are denied access to the room. Developers act like code monkeys, throwing the code over a metaphorical wall for the QA to test and “DevOps” to run. In “you build it,
このブログを書いてる経緯 rust.tokyo 楽しみ！絶対行く！といってたのに申込みを忘れたところ、じゃあスタッフとしてブログを書けという話になったので、ブロガー枠？らしく感想を書きます。とはいえ書けるのは見たやつだけです。 https://rust.tokyo/sessions# 前提 自分は低レベルプログラミングは詳しくないです。年に3日ぐらい思い出したように Rust 勉強することがある。 wasm 周りのエコシステムはずっと追ってる。 会場の雰囲気 組み込み勢とブロックチェーン勢が多そうな気配を感じた。 Visualization of mechanical CAD drawings using WebAssembly and WebGL Aki / CADDi (発表資料見つからず) 概要 Computer aided design (CAD) models used in m
Secrets from the Algorithm: Google Search’s Internal Engineering Documentation Has Leaked
Google, if you’re reading this, it’s too late. Ok. Cracks knuckles. Let’s get right to it. Internal documentation for Google Search’s Content Warehouse API has leaked. Google’s internal microservices appear to mirror what Google Cloud Platform offers and the internal version of documentation for the deprecated Document AI Warehouse was accidentally published publicly to a code repository for the c
As someone makes more money, expenses once considered luxuries can suddenly become seen as necessities: It’s called lifestyle creep. In the world of software development, we can suffer from a similar affliction: stack creep. Where hardware limitations once restricted developers to a minimalist approach, increased processing power, memory, and storage have led many down a more maximalist path. It’s
This article was first published on Cloudflare blog: When TCP sockets refuse to die Accompanying scripts While working on our Spectrum server, we noticed something weird: the TCP sockets which we thought should have been closed were lingering around. We realized we don't really understand when TCP sockets are supposed to time out! In our code, we wanted to make sure we don't hold connections to de
Why Twitter Didn’t Go Down: From a Real Twitter SRE
Twitter supposedly lost around 80% of its work force. What ever the real number is, there are whole teams with out engineers on it now. Yet, the website goes on and the tweets keep coming. This left a lot wondering what exactly was going on with all those engineers and made it seem like it was all just bloat. I’d like to explain my little corner of Twitter (though it wasn’t so little) and some of
SECURITY IS AWESOME SECURITY IS AWESOME I write about security and privacy. I regularly post original security research, custom tools, and detailed technical guides. Companies selling "security scorecards" are on the rise, and have started to become a factor in enterprise sales. I have heard from customers who were concerned about purchasing from suppliers who had been given poor ratings, and in a
This article was discussed on Hacker News. I recently learned of csvquote, a tool that encodes troublesome CSV characters such that unix tools can correctly process them. It reverses the encoding at the end of the pipeline, recovering the original input. The original implementation handles CSV quotes using the straightforward, naive method. However, there’s a better approach that is not only simpl
Linux perf Examples
Recent posts: 24 Mar 2024 » Linux Crisis Tools 17 Mar 2024 » The Return of the Frame Pointers 10 Mar 2024 » eBPF Documentary 28 Apr 2023 » eBPF Observability Tools Are Not Security Tools 01 Mar 2023 » USENIX SREcon APAC 2022: Computing Performance: What's on the Horizon 17 Feb 2023 » USENIX SREcon APAC 2023: CFP 02 May 2022 » Brendan@Intel.com 15 Apr 2022 » Netflix End of Series 1 09 Apr 2022 » Te
Go: A Documentary
Go: A Documentary by Changkun Ou <changkun.de> (and many inputs from contributors) This document collects many interesting (publicly observable) issues, discussions, proposals, CLs, and talks from the Go development process, which intends to offer a comprehensive reference of the Go history. Disclaimer Most of the texts are written as subjective understanding based on public sources Factual and ty
macOS Internals
macOS Internals.md macOS Internals Understand your Mac and iPhone more deeply by tracing the evolution of Mac OS X from prelease to Swift. John Siracusa delivers the details. Starting Points How to use this gist You've got two main options: Under Highlights, read all the links you're interested in, or Use "OS X Reviewed" as an index and just read all the reviews end to end. (This is not the fast o
ProductWork with GitHub Actions in your terminal with GitHub CLIgh brings GitHub to the command line by helping developers manage pull requests, issues, gists, and much more. As of 1.9.0, even more of GitHub is available in your terminal:… gh brings GitHub to the command line by helping developers manage pull requests, issues, gists, and much more. As of 1.9.0, even more of GitHub is available in
I joined GitLab in October 2015, and left in December 2021 after working there for a little more than six years. While I previously wrote about leaving GitLab to work on Inko, I never discussed what it was like working for GitLab between 2015 and 2021. There are two reasons for this: I was suffering from burnout, and didn't have the energy to revisit the last six years of my life (at that time)I w
A common debate in software development projects is between spending time on improving the quality of the software versus concentrating on releasing more valuable features. Usually the pressure to deliver functionality dominates the discussion, leading many developers to complain that they don't have time to work on architecture and code quality. Betteridge's Law of headlines is an adage that says
Screaming ArchitectureEvolution of a React folder structure and why to group by features right away React folder structures have been debated for years due to React's unopinionated approach, leading developers to ask, "Where should I put my files? How should I organize my code?" I've researched the most popular approaches to organizing React projects: Grouping by file type like components, context
Parse, don’t validate
Historically, I’ve struggled to find a concise, simple way to explain what it means to practice type-driven design. Too often, when someone asks me “How did you come up with this approach?” I find I can’t give them a satisfying answer. I know it didn’t just come to me in a vision—I have an iterative design process that doesn’t require plucking the “right” approach out of thin air—yet I haven’t bee
Before there was GraphQL, there was REST. In recent years, REST has become the dominant API style for building backend web services. With REST, you could signal the type of request we want to make (ex: GET, POST, PUT, or DELETE) and the resource we’d like to fetch or interact with (ex: /api/pets/1) using an HTTP method and a URL. It’s a great approach (and one we initially used at StockX for sever
The “Build Your Own Redis” Book is Completed | Blog | build-your-own.org
Read it here. Introduction Needless to say, the Redis project is quite a success. It’s an important component in backend applications. Redis could be considered one of the building blocks of modern computing. There are not many projects that fit the such role and stood the test of time. Here are some examples that meet my criteria of the “building block”: NGINX, SQLite, PostgreSQL, Kafka, Linux ke
L．フロリディのInformation: A Very Short Introductionがついに翻訳された。 邦題は『情報の哲学のために』である。 原書は2010年にOxfordのVery Short Introductionsシリーズから発売され、11年後の今になってようやく翻訳がなされたわけである。嬉しい。 フロリディの翻訳と言えば今年2月に何かが発売されたような気がしないでもないが、あれは錯覚である。 みんなして同じ悪い夢でも見ていたんだろう、きっと。 その錯覚のせいなのか「フロリディの翻訳」と聞くと若干身構えてしまうが、今回の翻訳はまともである。 翻訳を担当したのは図書館情報学や社会情報学などが専門の塩崎氏と河島氏だ。 情報の哲学が専門ではないものの、わりと近い分野からの翻訳ということで、幻となってしまった彼よりかはよっぽど専門家である。 そして、なんと言っても天下の勁草書房か
Speculation in JavaScriptCore
This post is all about speculative compilation, or just speculation for short, in the context of the JavaScriptCore virtual machine. Speculative compilation is ideal for making dynamic languages, or any language with enough dynamic features, run faster. In this post, we will look at speculation for JavaScript. Historically, this technique or closely related variants has been applied successfully t
Peter Jay Salzman, Michael Burian, Ori Pomerantz, Bob Mottram, Jim Huang 1 Introduction 1.1 Authorship 1.2 Acknowledgements 1.3 What Is A Kernel Module? 1.4 Kernel module package 1.5 What Modules are in my Kernel? 1.6 Is there a need to download and compile the kernel? 1.7 Before We Begin 2 Headers 3 Examples 4 Hello World 4.1 The Simplest Module 4.2 Hello and Goodbye 4.3 The __init and __exit Mac
August 8, 2019 // Jef Spaleta Sensu, Inc. This post walks you through how Kubernetes works, breaking down the individual components to illustrate how K8s helps you manage the lifecycle of containerized applications. DevOps It’s no secret that the popularity of running containerized applications has exploded over the past several years. Being able to iterate and release an application by provisioni
While working on our Spectrum server, we noticed something weird: the TCP sockets which we thought should have been closed were lingering around. We realized we don't really understand when TCP sockets are supposed to time out! In our code, we wanted to make sure we don't hold connections to dead hosts. In our early code we naively thought enabling TCP keepalives would be enough... but it isn't. I
MySQL InnoDBにおけるPKにUUIDを使ったINSERTのパフォーマンスの調査 - CubicLouve
下記の記事を見て、PKにUUIDを使った際に内部的にどうなっているのかを確認してみました kccoder.com 比較対象として、PKにULIDを使った場合も調べてみました。 github.com ULIDはUUIDと互換性がある、ソート可能な識別子です。 MySQLのバージョン % mysql --version mysql Ver 8.0.19 for osx10.14 on x86_64 (Homebrew) スキーマ mysql> SHOW CREATE TABLE innodb_auto_increment\G *************************** 1. row *************************** Table: innodb_auto_increment Create Table: CREATE TABLE `innodb_auto_incr
How to debug C and C++ programs with rr | Red Hat Developer
The common theme in many time-travel movies is to go back in time to find out what went wrong and fix it. Developers also have that desire to go back in time and find why the code broke and fix it. But, often, that crucial step where everything went wrong happened long ago, and the information is no longer available. The rr project lets programmers examine the entire life of a C or C++ program run
One of the cool new features in py-spy is the ability to profile native Python extensions written in languages like C, C++ or Cython. Almost all other Python profilers[1] only show program activity that is in pure Python code, and native code will instead show up as spending time in the line of Python that calls the native function. Using native profiling tools like perf can get you a sense of wha
TypeScript style guideThis is the style guide for the TypeScript language that was based on the one that is provided by Google. It contains both rules and best practices. Choose those that work best for your team. If you have a suggestion on how to improve this style guide please check the Changing the style guide below. This Style Guide uses RFC 2119 terminology when using the phrases must, must
Capture By AuthorTypescript has had Template Literals since its early stages. They are really useful when you want to create types from a static string. It was not until the Typescript 4.1 release that we saw Template Literal Types. In further releases, the Typescript team has been polishing its features and fixing some quirks. As a result, it is now a mature feature. What are Template Literal Typ
James Lindenbaum Szymon Kaliski Joshua Horowitz November 2022 With pen and paper, anyone can write a journal entry, draw a diagram, perform a calculation, or sketch a cartoon. Digital tablets like the iPad or reMarkable can adapt pen and paper into the world of digital media. In doing so, they trade away some of paper’s advantages like cheapness and tangibility. In exchange, we get new computation
When you’re accessing services over the WEB – let’s pick GMail as an example – couple of things have to happen upfront: The server you’re connecting to (GMail in our example) has to get to know who you are. Only after getting to know who you are it’s able to decide what resources you are allowed to access (e.g. your own email inbox, your Calendar, Drive etc.). Step 1 above is called authentication
RFC 9114: HTTP/3
Stream: Internet Engineering Task Force (IETF) RFC: 9114 Category: Standards Track Published: June 2022 ISSN: 2070-1721 Author: RFC 9114 HTTP/3 Abstract The QUIC transport protocol has several features that are desirable in a transport for HTTP, such as stream multiplexing, per-stream flow control, and low-latency connection establishment. This document describes a mapping of HTTP semantics over Q
My favourite Git commit
I like Git commit messages. Used well, I think they’re one of the most powerful tools available to document a codebase over its lifetime. I’d like to illustrate that by showing you my favourite ever Git commit. This commit is from my time at the Government Digital Service, working on GOV.UK. It’s from a developer by the name of Dan Carley, and it has the rather unassuming name of “Convert template
Avoiding dropped connections in nginx containers with “STOPSIGNAL SIGQUIT” | Ubuntu
Thank you for signing up for our newsletter! In these regular emails you will find the latest updates about Ubuntu and upcoming events where you can meet our team.Close (Also published on my blog at robinwinslow.uk) Update: The default used in the official nginx docker image was changed from SIGTERM to SIGQUIT in November 2020, so this should no longer be an issue for Docker or Kubernetes users. n
Automating the end-to-end lifecycle of Machine Learning applications Machine Learning applications are becoming popular in our industry, however the process for developing, deploying, and continuously improving them is more complex compared to more traditional software, such as a web service or a mobile application. They are subject to change in three axis: the code itself, the model, and the data
Have you heard about algebraic effects? My first attempts to figure out what they are or why I should care about them were unsuccessful. I found a few pdfs but they only confused me more. (There’s something about academic pdfs that makes me sleepy.) But my colleague Sebastian kept referring to them as a mental model for some things we do inside of React. (Sebastian works on the React team and came
Introducing the Cloud Development Kit for Terraform (Preview) | Amazon Web Services
AWS Developer Tools Blog Introducing the Cloud Development Kit for Terraform (Preview) Infrastructure as Code (IaC) is a fundamental component of modern DevOps practices because it enables you to deploy any version of your application infrastructure at will, and facilitates the full lifecycle management of all the resources required to run and monitor your application. Organizations who have adopt
EngineeringMaking GitHub’s new homepage fast and performantThis post is the third installment of our five-part series on building GitHub’s new homepage: How our globe is built How we collect and use the data behind the globe… This post is the third installment of our five-part series on building GitHub’s new homepage: How our globe is built How we collect and use the data behind the globe How we m
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
DBMSをGoで実装してみた - Sansan Tech Blog
DALL-E3 (ダリスリー) の無料教科書:初級編｜プチpony
REST vs. GraphQL vs. gRPC · Dan Hacks
rust.tokyo のまとめ・感想 - mizchi's blog
Marie Kondo your software stack with open source
When TCP sockets refuse to die — Idea of the day
HTTP Security Headers - A Complete Guide
Fast CSV processing with SIMD
Work with GitHub Actions in your terminal with GitHub CLI
What it was like working for GitLab
Is High Quality Software Worth the Cost?
Popular React Folder Structures and Screaming Architecture
Why use GraphQL? | Apollo GraphQL Blog
【書評】『情報の哲学のために』 - 榎本の雑感
The Linux Kernel Module Programming Guide
Sensu | How Kubernetes works
When TCP sockets refuse to die
Profiling Native Python Extensions
TS Style Guide
Mastering TypeScript Template Literal Types
Inkbase: Programmable Ink
How does Google Authenticator work? (Part 1)
Continuous Delivery for Machine Learning
Algebraic Effects for the Rest of Us
Making GitHub’s new homepage fast and performant