Bulk insertでも20時間以上かかっていたMySQLへのインサート処理を1時間以内にする - エムスリーテックブログ
この記事はエムスリー Advent Calendar 2022の30日目の記事です。 前日は id:kijuky による チームメンバーのGoogleカレンダーの休暇予定一覧をスプレッドシート+GASで作った でした。 AI・機械学習チームの北川(@kitagry)です。 今回はMySQLへのインサートを20倍以上高速化した話について書きます。 仕事をちゃんとしてるか見張る猫 TL; DR はじめに 今回のテーブル バイナリログを無効化する 追試 LOAD DATA INFILE 追試 テーブルの正規化 インデックスを一時的に剥がす まとめ We are hiring!! TL; DR バイナリログをオフにする LOAD DATA INFILEを使う インデックスを一時的に消す はじめに AI・機械学習チームではサイトトップからアプリに至るまで多くの推薦システムがあります。 そこでは推薦ロ
「Postgres で試した？」と聞き返せるようになるまでもしくはなぜ私は雰囲気で技術を語るのか？ — Just use Postgres 読書感想文 - じゃあ、おうちで学べる
はじめに 「Just use Postgres」という言葉を初めて聞いたのは、いつだったか覚えていません。Twitter か Hacker News か、あるいは社内の Slack か。どこで聞いたにせよ、私の反応は決まっていました。「また極端なことを言う人がいる」と。 「それ、〇〇でもできますよ」——この手のフレーズはもう100回は聞いてきました。そして大抵の場合、その〇〇は専用ツールに置き換えられていきます。技術が専門分化していくのは自然な流れです。 全文検索なら Elasticsearch。時系列データなら InfluxDB。メッセージキューなら RabbitMQ。それぞれの分野に専門家がいて、専用のソリューションがあって、ベストプラクティスがあります。「とりあえず Postgres で」なんて、それは思考停止ではないか、と。でも、心のどこかで気になっていたんです。 www.mann
GPT-5 の新パラメータとツール｜npaka
以下の記事が面白かったので、簡単にまとめました。 ・GPT-5 New Params and Tools - OpenAI Cookbook 1. verbosity1-1. 概要「verbosity」は、出力トークン数を調節できます。 ・low : 簡潔なUX、簡潔な文章 ・medium (デフォルト) : バランスの取れた詳細 ・high : 詳細な情報。監査、教育、引き継ぎに最適 1-2. verbosityの効果の確認プロンプトを一定に保ったまま、「verbosity」を変更することで、効果を確認できます。 response = client.responses.create( model="gpt-5", input="人生、宇宙、そして万物に関する究極の問いに対する答えは何でしょうか?", text={ "verbosity": "low" } ) print(response
Beyond exception handling, there's something else I see people struggling with, which is logging. Most people don't know what to log, so they decide to log anything thinking it might be better than nothing, and end up creating just noise. Noise is a piece of information that doesn't help you or your team understand what's going on or resolving a problem. Furthermore, I feel people are uncertain ab
MicrosoftのGraphRAG+Neo4jで簡単にKnowledge Graphを可視化する - Taste of Tech Topics
こんにちは。データサイエンティストチームYAMALEXのSsk1029Takashiです。 LLMが広く使用されるようになってから、RAGに関する研究も増加し、RAGを拡張する手法の研究も多く出ました。 その中でもテキストをグラフデータとして扱いRAGの元データとする手法をGraphRAGと言います。 今回はその中の一つであるMicrosoftが提唱しているGraphRAGの手法を使っていきます。 www.microsoft.com MicrosoftからはGraphRAGを使用できる実装もPythonライブラリとして発表されています。 ライブラリではグラフデータはあくまでRAGの元データとして使用していますが、せっかくテキストからグラフが生成できるのであれば、テキストマイニングしてみたくなるものです。 そこで、本記事ではこの手法で作成したグラフをグラフDBの一つであるNeo4jに投入して
> BTC: bc1qs0sq7agz5j30qnqz9m60xj4tt8th6aazgw7kxr ETH: 0x1D834755b5e889703930AC9b784CB625B3cd833E USDT(Tron): TPrCq8LxGykQ4as3o1oB8V7x1w2YPU2o5n Ton: UQAtBuFWI3H_LpHfEToil4iYemtfmyzlaJpahM3tFSoxomYQ Doge: D7GMQdKhKC9ymbT9PtcetSFTQjyPRRfkwTdismiss OOP: the worst thing that happened to programming [2/24/2025] In this article, we will try to understand why OOP is the worst thing that happened to prog
Introducing Ezno
Ezno is an experimental compiler I have been working on and off for a while. In short, it is a JavaScript compiler featuring checking, correctness and performance for building full-stack (rendering on the client and server) websites. This post is just an overview of some of the features I have been working on which I think are quite cool as well an overview on the project philosophy ;) It is still
A few months ago, I set myself the challenge of writing a C compiler in 500 lines of Python1, after writing my SDF donut post. How hard could it be? The answer was, pretty hard, even when dropping quite a few features. But it was also pretty interesting, and the result is surprisingly functional and not too hard to understand! There's too much code for me to comprehensively cover in a single blog
ChatGPT (Advanced Data Analysis)でBNF式からパーサーを生成する - Taste of Tech Topics
こんにちは、最近ピアノを習い始めた安部です。 今回は、ChatGPTのAdvanced Data Analysis (旧Code Interpreter)にBNF式を与えてパーサーを作成してもらおうと思います。 BNF式のように機械的に解釈可能なものであれば、正確にコードを生成してくれるのではないでしょうか？ BNFでうまくいけば、その他の様々な形式のデータやフォーマットからパーサーを自動生成してくれることが期待できそうです。 1. BNFとは BNF（バッカス・ナウア記法）とは、プログラムの構文規則（文脈自由文法）を記述するための記法です。 正確な定義よりも具体例を見た方が早く理解できると思うので、例を示します。 『プログラム意味論』（横内寛文 著）の冒頭に登場する、非常に単純なプログラムを許容する言語の定義です。 <変数> ::= A | B | C | ... | Z <定数> ::
Skills are custom instructions that extend Claude's capabilities for specific tasks or domains. When you create a skill via a SKILL.md file, you're teaching Claude how to handle specific scenarios more effectively. The power of skills lies in their ability to encode institutional knowledge, standardize outputs, and handle complex multi-step workflows that would otherwise require repeated explanati
Rewriting the Ruby parser
At Shopify, we have spent the last year writing a new Ruby parser, which we’ve called YARP (Yet Another Ruby Parser). As of the date of this post, YARP can parse a semantically equivalent syntax tree to Ruby 3.3 on every Ruby file in Shopify’s main codebase, GitHub’s main codebase, CRuby, and the 100 most popular gems downloaded from rubygems.org. We recently got approval to merge this work into C
Today we’re excited to announce the release of TypeScript 4.8! If you’re not yet familiar with TypeScript, it’s a language that builds on JavaScript and adds syntax for types. These types let you put your expectations and assumptions into your code, and those assumptions can then be checked by the TypeScript type-checker. This checking can help avoid typos, calling uninitialized values, mixing up
Golang Mini Reference 2022: A Quick Guide to the Modern Go Programming Language (REVIEW COPY)
Golang Mini Reference 2022 A Quick Guide to the Modern Go Programming Language (REVIEW COPY) Harry Yoon Version 0.9.0, 2022-08-24 REVIEW COPY This is review copy, not to be shared or distributed to others. Please forward any feedback or comments to the author. • feedback@codingbookspress.com The book is tentatively scheduled to be published on September 14th, 2022. We hope that when the release da
Version 1.108 is now available! Read about the new features and fixes from December. Release date: June 12, 2025 Security update: The following extension has security updates: ms-python.python. Update 1.101.1: The update addresses these issues. Update 1.101.2: The update addresses these issues. Downloads: Windows: x64 Arm64 | Mac: Universal Intel silicon | Linux: deb rpm tarball Arm snap Welcome t
I Made Zig Compute 33 Million Satellite Positions in 3 Seconds. No GPU Required.
Update: I've since added multithreading and pushed astroz to 326M propagations/sec. Read the follow-up → I've spent the past month optimizing SGP4 propagation and ended up with something interesting: astroz is now the fastest general purpose SGP4 implementation I'm aware of, hitting 11-13M propagations per second in native Zig and ~7M/s through Python with just pip install astroz. This post breaks
Parsing SQL - Strumenta
The code for this tutorial is on GitHub: parsing-sql SQL is a language to handle data in a relational database. If you worked with data you have probably worked with SQL. In this article we will talk about parsing SQL. It is in the same league of HTML: maybe you never learned it formally but you kind of know how to use it. That is great because if you know SQL, you know how to handle data. However
syntaxdesign
One of the most recognizable features of a languages is its syntax. What are some of the things about syntax that matter? What questions might you ask if you were creating a syntax for your own language? Motivation A programming language gives us a way structure our thoughts. Each program, has a kind of internal structure, for example: How can we capture this structure? One way is directly, via pi
I thought it would be fun to go outside of my comfort zone of web development topics and write about something completely different and new, something I have never written about before. So today, I'm going to show you how to implement a programming language! The project will parse and execute programs written in a simple language I called my (I know it's a lame name, but hey, it is "my" language).
Agentic GraphRAG for Commercial Contracts | Towards Data Science
In every business, legal contracts are foundational documents that define the relationships, obligations, and responsibilities between parties. Whether it’s a partnership agreement, an NDA, or a supplier contract, these documents often contain critical information that drives decision-making, risk management, and compliance. However, navigating and extracting insights from these contracts can be a
Over the course of my Spring 2020 semester at Harvey Mudd College, I developed a self-hosting compiler entirely from scratch. This article walks through many interesting parts of the project. It’s laid out so you can just read from beginning to end, but if you’re more interested in a particular topic, feel free to jump there. Or, take a look at the project on GitHub. Table of contents What the pro
A Walk with LuaJIT
The following is a chronicle of implementing a general purpose zero-instrumentation BPF based profiler for LuaJIT. Some assumptions are made about what this entails and it may be helpful to read some of our other work in this area. One major change from prior efforts is that instead of working with the original Parca unwinder we are now working with the OpenTelemetry eBPF profiler. If you missed t
Today we’re announcing our beta release of TypeScript 4.8! To get started using the beta, you can get it through NuGet, or- use npm with the following command: npm install -D typescript@beta You can also get editor support by Downloading for Visual Studio 2022/2019 Following directions for Visual Studio Code. Here’s a quick list of what’s new in TypeScript 4.8! Improved Intersection Reduction, Uni
The OpenSSL punycode vulnerability (CVE-2022-3602): Overview, detection, exploitation, and remediation | Datadog Security Labs
emerging threats and vulnerabilities The OpenSSL punycode vulnerability (CVE-2022-3602): Overview, detection, exploitation, and remediation November 1, 2022 emerging vulnerability On November 1, 2022, the OpenSSL Project released a security advisory detailing a high-severity vulnerability in the OpenSSL library. Deployments of OpenSSL from 3.0.0 to 3.0.6 (included) are vulnerable and are fixed in
こんにちは。Fintech SREの佐藤隆広(@T)です。 この記事は、Merpay & Mercoin Tech Openness Month 2025 の11日目の記事です。 Google社が提唱し、Site Reliability Engineering Bookによって広く知られるようになったSREの信頼性マネジメントは、開発と運用の関係性を再定義し、SLI/SLOとエラーバジェットに始まり、Availability・Latency・エラーレート・トラフィック・リソース飽和度・耐久性といったような指標で補強されてきました。 ところが近年、大規模言語モデル（LLM）の進歩が著しく、サービスにLLMを利用する機会が増えることによって、 プロンプトを数行変えただけで回答品質が変動する Latencyやエラーレートが良好でも幻覚（ハルシネーション）が急増する モデルの軽微なアップデートで回
The joy of building a ray tracer, for fun, in Rust. // flurries of latent creativity
TLDR? You can find the code and a bunch of examples on GitHub at dps/rust-raytracer. Over the holiday break, I decided to learn Rust. Rust is a modern systems programming language which has a really interesting type system. The type system can catch broad classes of common programming mistakes - e.g. ensuring memory is accessed safely - at compile time while generating tight, performant machine co
Tier 4 Support § Support for these targets is entirely experimental. If this target is provided by LLVM, LLVM may have the target as an experimental target, which means that you need to use Zig-provided binaries for the target to be available, or build LLVM from source with special configure flags. zig targets will display the target if it is available. This target may be considered deprecated by
Edge AI Just Got Faster
When Meta released LLaMA back in February, many of us were excited to see a high-quality Large Language Model (LLM) become available for public access. Many of us who signed up however, had difficulties getting LLaMA to run on our edge and personal computer devices. One month ago, Georgi Gerganov started the llama.cpp project to provide a solution to this, and since then his project has been one o
Version 1.108 is now available! Read about the new features and fixes from December. Release date: May 8, 2025 Update: Enable Next Edit Suggestions (NES) by default in VS Code Stable (more...). Update 1.100.1: The update addresses these security issues. Update 1.100.2: The update addresses these issues. Update 1.100.3: The update addresses these issues. Downloads: Windows: x64 Arm64 | Mac: Univers
⇦ 2024-10-17 Adding syntax to the cpython interpreter Condensed version of this cool blog post. Let's add some new syntax to Python! Making a small change is not so hard. Our aim is to make ternary statements default to None as they do in Ruby: >>> "hello" if 2 + 2 == 4 "hello" >>> "hello" if 2 + 2 == 5 None In existing Python, we get an error: File "<python-input-0>", line 1 "hello" if 2 + 2 == 5
Date: June 23, 2023 | Estimated Reading Time: 31 min | Author: Lilian Weng Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerfu
Solving Quantitative Reasoning Problems With Language Models
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz∗, Anders Andreassen†, David Dohan†, Ethan Dyer†, Henryk Michalewski†, Vinay Ramasesh†, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur∗, Guy Gur-Ari∗, and Vedant Misra∗ Google Research Abstract Language models have achieved remarkable performance on a wide range of tasks that require
Today we’re excited to announce our Release Candidate (RC) of TypeScript 4.8. Between now and the stable release of TypeScript 4.8, we expect no further changes apart from critical bug fixes. To get started using the RC, you can get it through NuGet, or use npm with the following command: npm install -D typescript@rc You can also get editor support by Downloading for Visual Studio 2022/2019 Follow
日本語で記述できるプログラミング言語「お抹茶」を作りながら学ぶインタプリタの仕組み - カミナシ エンジニアブログ
アイキャッチ画像 「日本語で記述できるプログラミング言語「お抹茶」を作りながら学ぶインタプリタの仕組み」 はじめに プログラミング言語って自分で作れるの？と思っていた僕ですが、実はインタプリタ型の簡単な言語なら意外と作れることがわかりました。 今回は「お抹茶」という日本語で書けるプログラミング言語を作ってみたので、その実装方法を紹介します。 インタプリタ言語を作るのに必要な3つの要素 プログラミング言語を作るというと難しそうですが、実は基本的な仕組みは意外とシンプルです。インタプリタ型言語を作るには、以下の3つの要素が必要になります。 1. トークナイザー（字句解析器） ソースコードを「トークン」と呼ばれる最小単位に分解する部分です。例えば 定義 x = 10 というコードを ["定義", "x", "=", "10"] のような配列に変換します。人間が読む文章を単語単位に区切るようなイメ
Compiling typed Python
It’s been nine whole years since PEP 484 landed and brought us types from on high. This has made a lot of people very angry and been widely regarded as a bad move1. Since then, people on the internet have been clamoring to find out: does this mean we can now compile Python to native code for more speed? It’s a totally reasonable question. It was one of my first questions when I first started worki
Tier 4 Support § Support for these targets is entirely experimental. If this target is provided by LLVM, LLVM may have the target as an experimental target, which means that you need to use Zig-provided binaries for the target to be available, or build LLVM from source with special configure flags. zig targets will display the target if it is available. This target may be considered deprecated by
The Andrej Karpathy episode. Andrej explains why reinforcement learning is terrible (but everything else is much worse), why model collapse prevents LLMs from learning the way humans do, why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education. Watch on YouTube; listen on Apple Podcasts or Spotify
I see people make the same mistakes over and over again when learning Rust. Here are my thoughts (ordered by importance) on how you can ease the learning process. My goal is to help you save time and frustration. Let Your Guard Down Stop resisting. That’s the most important lesson. Accept that learning Rust requires adopting a completely different mental model than what you’re used to. There are a
New – Amazon CloudWatch Evidently – Experiments and Feature Management | Amazon Web Services
AWS News Blog New – Amazon CloudWatch Evidently – Experiments and Feature Management Update Nov 29, 2021 – This post has been modified to provide more clarity on the new service. As a developer, I am excited to announce the availability of Amazon CloudWatch Evidently. This is a new Amazon CloudWatch capability that makes it easy for developers to introduce experiments and feature management in the
Python behind the scenes #13: the GIL and its effects on Python multithreading
As you probably know, the GIL stands for the Global Interpreter Lock, and its job is to make the CPython interpreter thread-safe. The GIL allows only one OS thread to execute Python bytecode at any given time, and the consequence of this is that it's not possible to speed up CPU-intensive Python code by distributing the work among multiple threads. This is, however, not the only negative effect of
Boring Python: code quality
Boring Python: code quality December 19, 2022 Django, Python This is the second in a series of posts I intend to write about how to build, deploy, and manage Python applications in as boring a way as possible. In the first post in the series I gave a definition of what I mean by “boring”, and it’s worth revisiting: I don’t mean “reliable” or “bug-free” or “no incidents”. While there is some overla
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Logging in Python like a PRO 🐍🌴
OOP: the worst thing that happened to programming
Writing a C compiler in 500 lines of Python
How to create Skills for Claude: steps and examples | Claude
Announcing TypeScript 4.8 - TypeScript
May 2025 (version 1.101)
Building a Toy Programming Language in Python
Kalyn: a self-hosting compiler for x86-64
Announcing TypeScript 4.8 Beta - TypeScript
SRE2.0: LLMサービスの信頼性を測る新しい評価指標の紹介 | メルカリエンジニアリング
0.8.0 Release Notes ⚡ The Zig Programming Language
April 2025 (version 1.100)
leontrolski - adding Python syntax
LLM Powered Autonomous Agents
Announcing TypeScript 4.8 RC - TypeScript
0.10.0 Release Notes ⚡ The Zig Programming Language
Andrej Karpathy — AGI is still a decade away
Flattening Rust's Learning Curve | corrode Rust Consulting