PlanetScale Postgres is the fastest way to run Postgres in the cloud. Plans start at just $5 per month. Learn more By Ben Dicken | September 9, 2024 What is a B-tree?The B-tree plays a foundational role in many pieces of software, especially database management systems (DBMS). MySQL, Postgres, MongoDB, Dynamo, and many others rely on B-trees to perform efficient data lookups via indexes. By the ti
Cloudflare Workers + Hono ワークショップ - ServerlessDays Tokyo 2023
Cloudflare Workers + Hono ワークショップ 資料はこちら => workshops.yusuke.run #serverlessdays Yusuke Wada 2023-09-24 ServerlessDays Tokyo 2023 workshops.yusuke.run アジェンダ ワークショップについて Workers イントロダクション Hono イントロダクション 基本編 プロキシ編 Web API編 フルスタック編 AI編 Honoをより深く知る その他 1. ワークショップについて 1.1 対象 対象者 Cloudflareでのアプリケーション作成に興味のある方 Honoを使ってみたい方 フロント、バックエンド問いません 前提条件 Wranglerが動く環境をつくっておく npx wrangler が動く JavaScriptに対する知識があるとよい
更新可能なデータレイクを構築するテーブルフォーマットApache Hudiについて - Repro Tech Blog
Reproでチーフアーキテクトを担当しているjoker1007です。 今回、社内のデータストレージの将来的な選択肢の一つとしてApache Hudiというテーブルデータフォーマットについて調査と実データでの検証を実施しました。 この記事では2回に分けて、そもそもhudiってどんなフォーマットなのか、どういうデータで検証してどんな結果が得られたのかについて紹介します。 ということで第1回は、hudiそのものについての紹介をしていきます。 この記事はhudi-0.14.1を利用して検証した時のものです。また社内向けに書いた資料の手直しであるため丁寧語でないことに御留意ください。 Hudiとは何か、その目的 hudiは更新可能なデータレイクを構築するためのテーブルフォーマットである。 ストリーミングによるデータインサートや、upsert, deleteをサポートする。 通常、データ分析に向いたデ
Joining CSV and JSON data with an in-memory SQLite database
Sponsored by: Honeycomb — AI agents behave unpredictably. Get the context you need to debug what actually happened. Read the blog 19th June 2021 The new sqlite-utils memory command can import CSV and JSON data directly into an in-memory SQLite database, combine and query it using SQL and output the results as CSV, JSON or various other formats of plain text tables. sqlite-utils memory The new feat
The Fediverse has been a hot topic of discussion lately, with thousands, if not millions, of new users creating accounts on platforms like Mastodon to either move entirely to "the other side" or experiment and learn about this new social network. Today we're introducing Wildebeest, an open-source, easy-to-deploy ActivityPub and Mastodon-compatible server built entirely on top of Cloudflare's Super
TL;DR: DuckDB-Wasm is an in-process analytical SQL database for the browser. It is powered by WebAssembly, speaks Arrow fluently, reads Parquet, CSV and JSON files backed by Filesystem APIs or HTTP requests and has been tested with Chrome, Firefox, Safari and Node.js. You can try it in your browser at shell.duckdb.org or on Observable. DuckDB-Wasm is fast! If you're here for performance numbers, h
TL;DR; We are changing std::sort in LLVM’s libcxx. That’s a long story of what it took us to get there and all possible consequences, bugs you might encounter with examples from open source. We provide some benchmarks, perspective, why we did this in the first place and what it cost us with exciting ideas from Hyrum’s Law to reinforcement learning. All changes went into open source and thus I can
Weird Lexical Syntax
I just learned 42 programming languages this month to build a new syntax highlighter for llamafile. I feel like I'm up to my eyeballs in programming languages right now. Now that it's halloween, I thought I'd share some of the spookiest most surprising syntax I've seen. The languages I decided to support are Ada, Assembly, BASIC, C, C#, C++, COBOL, CSS, D, FORTH, FORTRAN, Go, Haskell, HTML, Java,
しくじり先生のように学ぶ「NFS+sqliteで苦労した話から学ぶ、問題解決の考え方」という勉強会をやってみました | Raccoon Tech Blog [株式会社ラクーンホールディングス 技術戦略部ブログ]
株式会社ラクーンホールディングスのエンジニア/デザイナーから技術情報をはじめ、世の中のためになることや社内のことなどを発信してます。 インフラLinux勉強会SQLiteNFS こんにちは、羽山です。 今回は社内で実施した勉強会をほぼそのままブログ記事にしています。 ITエンジニアたるものドキュメントを残したり勉強会を実施したり分かりやすい設計にリファクタリングしたりなど、日々 知見の伝達 を意識していると思います。主要な知見はたいていそれらの方法で満たせるのですが、しかし残念ながら中には画一的な方法では伝えにくいものもあったりします。 ところで伝統芸能や工芸の世界では技術の継承において 背中を見て学ぶ という方法が用いられることがあるようです。 これと似た手法はITエンジニアの世界にもあって、先輩エンジニアとペアプログラミングをすることで文書では伝わりにくい技術や考え方・テクニックを学ぶ
![しくじり先生のように学ぶ「NFS+sqliteで苦労した話から学ぶ、問題解決の考え方」という勉強会をやってみました | Raccoon Tech Blog [株式会社ラクーンホールディングス 技術戦略部ブログ]](https://cdn-ak-scissors.b.st-hatena.com/image/square/adda0d926166e07709f641e0594d39801c8333d9/backend=imagemagick;height=288;version=1;width=512/https%3A%2F%2Ftechblog.raccoon.ne.jp%2Fwp-content%2Fuploads%2F2021%2F10%2Fscience_hakase_shippai_s.png)
The product https://joinmastodon.org Mastodon is a free, open-source social network server based on ActivityPub where users can follow friends and discover new ones. On Mastodon, users can publish anything they want: links, pictures, text, and video. All Mastodon servers are interoperable as a federated network. Open source The project is open source at https://github.com/mastodon/mastodon License
SQLite is the most deployed and most used database. There are over one trillion (1000000000000 or a million million) SQLite databases in active use. It is maintained by three people. They don’t allow outside contributions. SQLite is likely used more than all other database engines combined. Billions and billions of copies of SQLite exist in the wild. It’s everywhere. It is also probably one of the
Achieve a high-speed InnoDB purge on Amazon RDS for MySQL and Amazon Aurora MySQL | Amazon Web Services
AWS Database Blog Achieve a high-speed InnoDB purge on Amazon RDS for MySQL and Amazon Aurora MySQL Purge is a housekeeping operation in a MySQL database. The InnoDB storage engine relies on it to clean up undo logs and delete-marked table records that are no longer needed for multiversion concurrency control (MVCC) or rollback operations. While our applications pursue a database design that aims
【速報】Rails 8.0.0がリリースされました Rails 8.0（v8.0.0タグ）は、7.2.0リリースから半年後のリリースということもあり、メジャーリリースとしてはChangelogの変更件数が少なめなのが特徴です。機能面では、Strong Parametersのexpect追加や、MySQL 5.5のサポート廃止が目につきます。大きなbreaking changesも今のところ見当たりません。 rails/rails at v8.0.0 以下、Rails 8.0のChangelogを件数の多い順に見ていきます。なお、原則として削除のChangelogは、その項目が非推奨化されたときのプルリク情報にリンクしています。 なお、以下の機能にはChangelogがありません。 Action Mailbox Action Mailer 🔗 Active Record（56件） Rail
Developers who build Worker applications focus on what they're creating, not the infrastructure required, and benefit from the global reach of Cloudflare's network. Many applications require persistent data, from personal projects to business-critical workloads. Workers offer various database and storage options tailored to developer needs, such as key-value and object storage. Relational database
Error Handling In Rust - A Deep Dive May 13, 2021 8550 words 43 min This article is a sample from Zero To Production In Rust, a hands-on introduction to backend development in Rust. You can get a copy of the book at zero2prod.com. TL;DR To send a confirmation email you have to stitch together multiple operations: validation of user input, email dispatch, various database queries. They all have one
I want to troubleshoot low memory issues when I run an Amazon Relational Database Service (Amazon RDS) for MySQL instance. My available memory is low, my database is out of memory, or there are latency issues in my application. Resolution Important: Performance Insights will reach its end of life on June 30, 2026. You can upgrade to the Advanced mode of Database insights before June 30, 2026. If y
Against SQL
TLDR The relational model is great: A shared universal data model allows cooperation between programs written in many different languages, running on different machines and with different lifespans. Normalization allows updating data without worrying about forgetting to update derived data. Physical data independence allows changing data-structures and query plans without having to change all of y
Parquet and Postgres in the Data Lake | Crunchy Data Blog
Interested in Spatial analytics? You can now connect Postgres and PostGIS to CSV, JSON, Parquet / GeoParquet, Iceberg, and more with Crunchy Data Warehouse. Static Data is DifferentA couple weeks ago, I came across a blog from Retool on their experience migrating a 4TB database. They put in place some good procedures and managed a successful migration, but the whole experience was complicated by t
In 2006 Microsoft conducted a customer survey to find what new features users want in new versions of Microsoft Office. To their surprise, more than 90% of what users asked for already existed, they just didn't know about it. To address the "discoverability" issue, they came up with the "Ribbon UI" that we know from Microsoft Office products today. Office is not unique in this sense. Most of us ar
Local-first software: You own your data, in spite of the cloud
Cloud apps like Google Docs and Trello are popular because they enable real-time collaboration with colleagues, and they make it easy for us to access our work from all of our devices. However, by centralizing data storage on servers, cloud apps also take away ownership and agency from users. If a service shuts down, the software stops functioning, and data created with that software is lost. In t
はじめに こんにちは！J-CAT 株式会社でエンジニアをしている田中です。 弊社では、魅力あふれる日本の姿をここでしかできない感動体験として届ける予約サイト「Otonami」というサービスを運営しており、このサービスのバックエンドとして、Go言語を採用しております。 これまで課題として認識していながらも、中々着手できずにいたバックエンドのCI/CD改善に本格的に取り組み、その処理速度を劇的に向上させることができました。 この記事では、そのために実施したアプローチを汎用的に活用できる手法として整理し、紹介していきます。 結論として、ざっくり以下の対応を行いました。 ビルドタグを利用したジョブの並列化 デプロイ先の変更(不本意) txdbを利用したトランザクション → ロールバックのテスト t.Parallelを用いたテストの並列化 テストをdockerコンテナ実行 → プリセットアクション実
Database Fundamentals
About a year ago, I tried thinking which database I should choose for my next project, and came to the realization that I don't really know the differences of databases enough. I went to different database websites and saw mostly marketing and words I don't understand. This is when I decided to read the excellent books Database Internals by Alex Petrov and Designing Data-Intensive Applications by
Try Mastering Emacs for free! Are you struggling with the basics? Have you mastered movement and editing yet? When you have read Mastering Emacs you will understand Emacs. It’s that time again: there’s a new major version of Emacs and, with it, a treasure trove of new features and changes. Notable features include the formal inclusion of native compilation, a technique that will greatly speed up y
10 Things Software Developers Should Learn about Learning – Communications of the ACM
The dashed box on the left contains exactly the same information as the awkward textual description in the dashed box on the right. But if a developer only received one of the two to create an SQL database, they are likely to find the diagram easier than the text. We say that the text here has a higher extraneous cognitive load. When faced with a task that seems beyond a person’s abilities, it is
If you need a publish/subscribe or job server at any point in your project, try using Postgres. It'll give you lots of data integrity and performance guarantees, and it doesn't require you or your team learning any new technology. If you're making any project of sufficient complexity, you'll need a publish/subscribe server to process events. This article will introduce you to Postgres, explain the
Introduction to Go 1.22 The latest Go release, version 1.22, arrives six months after Go 1.21. Most of its changes are in the implementation of the toolchain, runtime, and libraries. As always, the release maintains the Go 1 promise of compatibility. We expect almost all Go programs to continue to compile and run as before. Changes to the language Go 1.22 makes two changes to “for” loops. Previous
Introduction to Go 1.23 The latest Go release, version 1.23, arrives six months after Go 1.22. Most of its changes are in the implementation of the toolchain, runtime, and libraries. As always, the release maintains the Go 1 promise of compatibility. We expect almost all Go programs to continue to compile and run as before. Changes to the language The “range” clause in a “for-range” loop now accep
Does OLAP need an ORM?
TL;DR · ORMs have proven to be useful for many developers in the OLTP/transactional stack (Postgres, MySQL, etc). · OLAP/analytical databases like ClickHouse could potentially benefit from ORM abstractions. · Existing transactional ORMs probably shouldn’t be extended to OLAP due to fundamental differences in semantic meaning between OLTP and OLAP. · Moose OLAP (part of MooseStack) is an open sourc
An AI-native software engineer is one who deeply integrates AI into their daily workflow, treating it as a partner to amplify their abilities. This requires a fundamental mindset shift. Instead of thinking “AI might replace me” an AI-native engineer asks for every task: “Could AI help me do this faster, better, or differently?”. The mindset is optimistic and proactive - you see AI as a multiplier
After learning about indexes, I understood their basic structure, but I wanted to dig deeper — to explore the data structure, understand the algorithm, and learn how the index data is stored on disk. The theory and actual implementation can differ, so I decided to explore this topic further. I wanted to see how a database management system (DBMS) stores an index in both disk and memory, and how it
Noble Numbat Release Notes Table of Contents Introduction New features in 24.04 LTS Known Issues Official flavours More information Introduction These release notes for Ubuntu 24.04 LTS (Noble Numbat) provide an overview of the release and document the known issues with Ubuntu and its flavours. For details of the changes applied since 24.04, please see the 24.04.2 change summary. Support lifespan
これは、なにをしたくて書いたもの？ Goを使って、データベースにアクセスするコードを書いてみたいなぁと思いまして。 sqlパッケージ Goでデータベースにアクセスするには、sqlパッケージを使うようです。 sql - The Go Programming Language sqlパッケージは、SQL（ライクな）データベースにアクセスするための、汎用インターフェースを提供するパッケージだそうです。 Package sql provides a generic interface around SQL (or SQL-like) databases. 基本的な使い方は、こちらを参照。 SQLInterface · golang/go Wiki · GitHub そして、sqlパッケージのインターフェースを実装したドライバーは、こちらの一覧で確認できます。 SQLDrivers · golang
Pandas is a very popular tool for data analysis. It comes built-in with many useful features, it's battle tested and widely accepted. However, pandas is not always the best tool for the job. SQL databases have been around since the 1970s. Some of the smartest people in the world worked on making it easy to slice, dice, fetch and manipulate data quickly and efficiently. SQL databases have come such
Horizontally scaling the Rails backend of Shop app with Vitess - Shopify
Horizontally scaling the Rails backend of Shop app with VitessShop app horizontally scaled a Ruby on Rails app with Vitess. This blog describes Vitess and our detailed approach for introducing Vitess to a Rails app. Good problems We experienced hockey stick growth after we launched the Shop app. We were glued to our dashboards and saw millions of users onboard onto the app. This was gratifying, bu
Cognitive load is what matters
The logo image was taken from Reddit. It is a living document, last update: May 2025. Your contributions are welcome! Introduction There are so many buzzwords and best practices out there, but most of them have failed. We need something more fundamental, something that can't be wrong. Sometimes we feel confusion going through the code. Confusion costs time and money. Confusion is caused by high co
After studying how companies deploy generative AI applications, I noticed many similarities in their platforms. This post outlines the common components of a generative AI platform, what they do, and how they are implemented. I try my best to keep the architecture general, but certain applications might deviate. This is what the overall architecture looks like. This is a pretty complex system. Thi
Concurrency control in Amazon Aurora DSQL | Amazon Web Services
AWS Database Blog Concurrency control in Amazon Aurora DSQL In any database management system (DBMS), concurrency control plays a crucial role in making sure multiple transactions can occur simultaneously without leading to inconsistencies or corrupt data. It refers to the mechanisms that manage how transactions interact, particularly when they access shared resources. A distributed database manag
Build incremental data pipelines to load transactional data changes using AWS DMS, Delta 2.0, and Amazon EMR Serverless | Amazon Web Services
AWS Big Data Blog Build incremental data pipelines to load transactional data changes using AWS DMS, Delta 2.0, and Amazon EMR Serverless Building data lakes from continuously changing transactional data of databases and keeping data lakes up to date is a complex task and can be an operational challenge. A solution to this problem is to use AWS Database Migration Service (AWS DMS) for migrating hi
You can execute database transactions using an sql.Tx, which represents a transaction. In addition to Commit and Rollback methods representing transaction-specific semantics, sql.Tx has all of the methods you use to perform common database operations. To get the sql.Tx, you call DB.Begin or DB.BeginTx. A database transaction groups multiple operations as part of a larger goal. All of the operation
Streamlining access to tabular datasets stored in Amazon S3 Tables with DuckDB | Amazon Web Services
AWS Storage Blog Streamlining access to tabular datasets stored in Amazon S3 Tables with DuckDB As businesses continue to rely on data-driven decision-making, there’s an increasing demand for tools that streamline and accelerate the process of data analysis. Efficiency and simplicity in application architecture can serve as a competitive edge when driving high-stakes decisions. Developers are seek
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
B-trees and database indexes — PlanetScale
Welcome to Wildebeest: the Fediverse on Cloudflare
DuckDB-Wasm: Efficient Analytical SQL in the Browser
Changing std::sort at Google’s Scale and Beyond
Mastodon: Ruby on Rails Open Source Web App
Collection of insane and fun facts about SQLite - blag
Rails 8.0 Changelog（全項目リンク付き）｜TechRacho by BPS株式会社
Building D1: a Global Database
Error Handling In Rust - A Deep Dive | Luca Palmieri
Troubleshoot low freeable memory in Amazon RDS for MySQL
Lesser Known PostgreSQL Features
GoアプリのCI/CDを4倍高速化した汎用的手法まとめ【txdb】
What's New in Emacs 28.1?
Postgres is a great pub/sub & job server
Go 1.22 Release Notes - The Go Programming Language
Go 1.23 Release Notes - The Go Programming Language
The AI-Native Software Engineer
SQLite Index Visualization: Structure
Ubuntu 24.04 LTS (Noble Numbat) Release Notes
GoでMySQLにアクセスしてみる - CLOVER🍀
Practical SQL for Data Analysis
Building A Generative AI Platform
Executing transactions - The Go Programming Language