本文の内容は、2024年11月25日に Alejandro Villanueva が投稿したブログ(https://sysdig.com/blog/26-aws-security-best-practices/)を元に日本語に翻訳・再構成した内容となっております。 セキュリティは、 AWS Foundational セキュリティベストプラクティスの基本的な柱です。セキュリティリスクを最小限に抑え、環境を保護するには、サービス別にまとめられた AWS セキュリティベストプラクティスに従うことが不可欠です。この構造化されたアプローチは、潜在的な脆弱性に積極的に対処し、堅牢で安全なクラウドアーキテクチャーを維持するのに役立ちます。 AWS IAM（1） IAMポリシーでは、フルの ” * ” 管理者権限を許可すべきではない （２） IAMユーザーにはIAMポリシーを添付してはならない （３） I
法律のデータ構造と検索
デジタル庁は、法令標準 XML スキーマに準拠した、現行の法令データをe-Gov法令検索というサイト上で公開しています[1]。今回、この法令XMLをパースするPythonライブラリ ja-law-parser をつくり、法令データの全文検索をしてみました。 この記事では、日本の法令とそのデータ構造、法令XMLパーサについて解説し、最後に、それらを使った法令データの全文検索システムを実装する方法をご紹介します。法令検索の実装についても、GitHubリポジトリで公開しています。 この記事は、情報検索・検索技術 Advent Calendar 2023の16日目の記事です。 法律と法令 法律とは 法律の制定と公布 法律と法令の違い 法律の改正 法令のデータ構造 e-Govの法令データ 法令標準XMLスキーマ 法令番号と法令ID 題名 本則と附則 条・項・号 編・章・節・款・目 法令XMLパーサ：
TabFS
Going through the files inside a tab's folder. For example, the url.txt, text.txt, and title.txt files tell me those live properties of this tab (Read more up-to-date documentation for all of TabFS's files here.) This gives you a ton of power, because now you can apply all the existing tools on your computer that already know how to deal with files -- terminal commands, scripting languages, point-
大実験！ChatGPTは競プロの問題を解けるのか (2024年5月版) - E869120's Blog
1. はじめに 2024 年 5 月 14 日、OpenAI 社から新たな生成 AI「GPT-4o」が発表され、世界に大きな衝撃を与えました。これまでの GPT-4 よりも性能を向上させただけでなく1、音声や画像のリアルタイム処理も実現し、さらに応答速度が大幅に速くなりました。「ついにシンギュラリティが来てしまったか」「まるで SF の世界を生きているような感覚だ」という感想も見受けられました。 しかし、いくら生成 AI とはいえ、競技プログラミングの問題を解くのは非常に難しいです。なぜなら競技プログラミングでは、問題文を理解する能力、プログラムを実装する能力だけでなく、より速く答えを求められる解法 (アルゴリズム) を考える能力も要求されるからです。もし ChatGPT が競技プログラミングを出来るようになれば他のあらゆるタスクをこなせるだろう、と考える人もいます。 それでは、現代最強の
This post is part of a collection on UUIDs. What is IDOR? Indirect Object Reference (IDOR) occurs when a resource can be accessed directly by its ID even when the user does not have proper authorization to access it. IDOR is a common mistake when using a separate service for storing files, such as a publicly readable Amazon S3 bucket. The web application may perform access control checks correctly
REST API Design Best Practices Handbook – How to Build a REST API with JavaScript, Node.js, and Express.js
By Jean-Marc Möckel I've created and consumed many API's over the past few years. During that time, I've come across good and bad practices and have experienced nasty situations when consuming and building API's. But there also have been great moments. There are helpful articles online which present many best practices, but many of them lack some practicality in my opinion. Knowing the theory with
Deploy applications on Amazon ECS using Docker Compose | Amazon Web Services
Containers Deploy applications on Amazon ECS using Docker Compose Note: Docker Compose’s integration with Amazon ECS has been deprecated and is retiring in November 2023 There are many reasons why containers have become popular since Docker democratized access to the core Linux primitives that make a “docker run” possible. One reason is that containers are not tied to a specific infrastructure or
[Hacker News discussion, LinkedIn discussion, Twitter thread] Update: My upcoming book, AI Engineering (late 2024/early 2025) will cover building aplications with foundation models in depth. A question that I’ve been asked a lot recently is how large language models (LLMs) will change machine learning workflows. After working with several companies who are working with LLM applications and persona
Performance comparison: counting words in Python, Go, C++, C, AWK, Forth, and Rust
Performance comparison: counting words in Python, Go, C++, C, AWK, Forth, and Rust March 2021 Summary: I describe a simple interview problem (counting frequencies of unique words), solve it in various languages, and compare performance across them. For each language, I’ve included a simple, idiomatic solution as well as a more optimized approach via profiling. Go to: Constraints | Python Go C++ C
This post is about why we need a coroutine package for Go, and what it would look like. But first, what are coroutines? Every programmer today is familiar with function calls (subroutines): F calls G, which stops F and runs G. G does its work, potentially calling and waiting for other functions, and eventually returns. When G returns, G is gone and F continues running. In this pattern, only one fu
Introducing Amazon Managed Workflows for Apache Airflow (MWAA) | Amazon Web Services
AWS News Blog Introducing Amazon Managed Workflows for Apache Airflow (MWAA) As the volume and complexity of your data processing pipelines increase, you can simplify the overall process by decomposing it into a series of smaller tasks and coordinate the execution of these tasks as part of a workflow. To do so, many developers and data engineers use Apache Airflow, a platform created by the commun
Introducing Ezno
Ezno is an experimental compiler I have been working on and off for a while. In short, it is a JavaScript compiler featuring checking, correctness and performance for building full-stack (rendering on the client and server) websites. This post is just an overview of some of the features I have been working on which I think are quite cool as well an overview on the project philosophy ;) It is still
A few months ago, I set myself the challenge of writing a C compiler in 500 lines of Python1, after writing my SDF donut post. How hard could it be? The answer was, pretty hard, even when dropping quite a few features. But it was also pretty interesting, and the result is surprisingly functional and not too hard to understand! There's too much code for me to comprehensively cover in a single blog
AWS Lambda（Python）でAmazon Bedrockの出力をレスポンスストリーミング対応してみた - Taste of Tech Topics
こんにちはイワツカです。 今年の夏は、特に猛暑日が続いていたので、例年にも増して素麺を食べてました。 さて今回は、AWS Lambda（Python）でLambda Web Adapterを用いてレスポンスストリーミングする方法を試してみたので紹介します。 1. 概要 1.1 レスポンスストリーミングとは？ 1.2 Lambda Web Adapterとは？ 2. アプリ作成 2.1 実行環境 2.2 ディレクトリ構成 2.3 FastAPIの実装 2.4 Streamlitの実装 3. アプリを動かして見る 3.1 チャットアプリをデプロイ 3.2 Streamlitを起動する 3.3 チャットを試してみる 4. まとめ 1. 概要 1.1 レスポンスストリーミングとは？ レスポンスストリーミングとは、HTTPリクエストに対してサーバーがレスポンスを一度にまとめて送るのではなく、データを
PostgreSQL 15 Released!
October 13, 2022 - The PostgreSQL Global Development Group today announced the release of PostgreSQL 15, the latest version of the world’s most advanced open source database. PostgreSQL 15 builds on the performance improvements of recent releases with noticeable gains for managing workloads in both local and distributed deployments, including improved sorting. This release improves the developer e
We Love You, TypeScriptFor nearly five years now, Motion has operated in a large TypeScript monorepo. At its peak, it was roughly ~2.5 million lines of code after excluding comments, node_modules, etc. To manage this, we used Vercel’s rather excellent Turborepo build system. This is not a blog post hating on TypeScript — quite the opposite! Motion would likely not even have survived until today wi
こちらはLayerX AI Agentブログリレー36日目の記事です。 LayerX バクラク事業部で AI/MLOpsエンジニアをしている中村(@po3rin)です。今回はAI Agentのビジネス価値を計るバックテスト基盤を構築した話と、そこから学んだAI Agent開発のプラクティスを紹介します。 目次 目次 AI Agent機能の評価の重要性 AI Agent機能のバックテスト バックテスト基盤開発の難しさ バックテスト基盤を実現する技術 全体構成 SnowflakeへのSnapshotデータアクセス LLM Native GORM Plugin「Firn」 なぜLLMで書き換えるのか GORM Pluginの作り方 LLMによるクエリ書き換え部分 LLMのクエリ書き換えガードレール SQL BASE Golden Testing Package「gormgolden」 今回のバッ
Rewriting the Ruby parser
At Shopify, we have spent the last year writing a new Ruby parser, which we’ve called YARP (Yet Another Ruby Parser). As of the date of this post, YARP can parse a semantically equivalent syntax tree to Ruby 3.3 on every Ruby file in Shopify’s main codebase, GitHub’s main codebase, CRuby, and the 100 most popular gems downloaded from rubygems.org. We recently got approval to merge this work into C
In 2021 Ben Bridts published a highly inventive method for finding the AWS Account ID of a public S3 bucket. This post describes a technique to find the Account ID of any S3 bucket (both private and public). I'd highly recommend reading Ben's technique first as we will re-use a lot of concepts. S3 Bucket to AWS Account IDShell output can be worth a thousand words, here's what our technique enables
Incident Metrics in SRE
Štěpán Davidovič Incident Metrics in SRE Critically Evaluating MTTR and Friends Boston Farnham Sebastopol Tokyo Beijing Boston Farnham Sebastopol Tokyo Beijing 978-1-098-10313-2 [LSI] Incident Metrics in SRE by Štěpán Davidovič Copyright © 2021 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebas
Introduction⌗ Postgres queue tech is a thing of beauty, but far from mainstream. Its relative obscurity is partially attributable to the cargo cult of “scale”. The scalability cult has decreed that there are several queue technologies with greater “scalability” than Postgres, and for that reason alone, Postgres isn’t suitably scalable for anyone’s queuing needs. The cult of scalability would rathe
June 5, 2023 by Dirk Bäumer VS Code for the Web (https://vscode.dev) has been available for some time now and it has always been our goal to support the full edit / compile / debug cycle in the browser. This is relatively easy for languages like JavaScript and TypeScript since browsers ship with a JavaScript execution engine. It is harder for other languages since we must be able to execute (and t
Introduction Data preprocessing is a necessary step in any machine learning workflow, affecting both the model’s effectiveness and the ease of maintenance. While scikit-learn is commonly used for preprocessing due to its integration with the broader Python ecosystem, DuckDB offers a practical alternative by enabling SQL-based data transformations within Python. Its declarative syntax supports modu
4 Pandas Anti-Patterns to Avoid and How to Fix Them
pandas is a powerful data analysis library with a rich API that offers multiple ways to perform any given data manipulation task. Some of these approaches are better than others, and pandas users often learn suboptimal coding practices that become their default workflows. This post highlights four common pandas anti-patterns and outlines a complementary set of techniques that you should use instea
February 2022 Recently I improved the performance of GoAWK – my AWK interpreter written in Go – by switching from a tree-walking interpreter to a bytecode compiler with a virtual machine interpreter. While doing that, I thought it’d be interesting to see how much the performance of Go itself has improved over the years. There are many ways programs written in Go have gotten faster: the Go team and
Photo by Claudio Schwarz on Unsplash. How to create a Python package? In order to create a Python package, you need to write the code that implements the functionality you want to put in your package, and then you need to publish it to PyPI. That is the bare minimum. Nowadays, you can also set up a variety of other things to make your life easier down the road: continuous testing of your package;
I don't build systems. I imagine them, then write them. …that were written in a military training camp and accidentally grew to 5k words. I am writing this on my phone, in a barrack that houses some 200+ of my brothers-in-arms in the Ukrainian army’s training camp; I use short periods of rest between training, mostly at night and on Sundays. TBH, since joining the army, I didn’t expect to have tim
Why Rust is the Future of Game Development | thefuntastic
Rust, not related to the video game also called Rust, is a promising systems programming language with novel features ideally suited for game development. Exposure and awareness within the game developer community, however, remains limited. In this post, I provide a gentle introduction to Rust and attempt to justify its place on your radar. A Short History LessonWhat is Rust, and where did it com
Ruby is an interpreted, open-source programming language with a focus on simplicity and productivity. Ruby is already 30 years old, and it has become one of the most used programming languages. Some claim Ruby is dead; others use it widely in their companies. Join me in this blog post as we learn about Ruby's history and where it stands today. How It StartedRuby was born on Feb. 24th, 1993. This w
Agents
Intelligent agents are considered by many to be the ultimate goal of AI. The classic book by Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (Prentice Hall, 1995), defines the field of AI research as “the study and design of rational agents.” The unprecedented capabilities of foundation models have opened the door to agentic applications that were previously unimaginabl
September 2021 Summary: Python 3.10, which is due out in early October 2021, will include a large new language feature called structural pattern matching. This article is a critical but (hopefully) informative presentation of the feature, with examples based on real-world code. Go to: What it is | Where it shines | My code | Other projects | Problems | Wrapping up At a recent local Python meetup,
Internet Engineering Task Force (IETF) K. Davis Request for Comments: 9562 Cisco Systems Obsoletes: 4122 B. Peabody Category: Standards Track Uncloud ISSN: 2070-1721 P. Leach University of Washington May 2024 Universally Unique IDentifiers (UUIDs) Abstract This specification defines UUIDs (Universally Unique IDentifiers) -- also known as GUIDs (Globally Unique IDentifiers) -- and a Uniform Resou
Weird Lexical Syntax
I just learned 42 programming languages this month to build a new syntax highlighter for llamafile. I feel like I'm up to my eyeballs in programming languages right now. Now that it's halloween, I thought I'd share some of the spookiest most surprising syntax I've seen. The languages I decided to support are Ada, Assembly, BASIC, C, C#, C++, COBOL, CSS, D, FORTH, FORTRAN, Go, Haskell, HTML, Java,
Prompt Engineering
Date: March 15, 2023 | Estimated Reading Time: 21 min | Author: Lilian Weng Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empirical science and the effect of prompt engineering methods can vary a lot among models, thus requiring heavy experimentation a
Golang Mini Reference 2022: A Quick Guide to the Modern Go Programming Language (REVIEW COPY)
Golang Mini Reference 2022 A Quick Guide to the Modern Go Programming Language (REVIEW COPY) Harry Yoon Version 0.9.0, 2022-08-24 REVIEW COPY This is review copy, not to be shared or distributed to others. Please forward any feedback or comments to the author. • feedback@codingbookspress.com The book is tentatively scheduled to be published on September 14th, 2022. We hope that when the release da
GitHub - cozodb/cozo: A transactional, relational-graph-vector database that uses Datalog for query. The hippocampus for AI!
Version v0.7: after HNSW vector search from 0.6, in 0.7 we bring to you MinHash-LSH for near-duplicate search, full-text search, Json value support and more! See here for more details. Version v0.6 released! This version brings vector search with HNSW indices inside Datalog, which can be integrated seamlessly with powerful features like ad-hoc joins, recursive Datalog and classical whole-graph alg
AST vs. Bytecode: Interpreters in the Age of Meta-Compilation
233 AST vs. Bytecode: Interpreters in the Age of Meta-Compilation OCTAVE LAROSE, University of Kent, UK SOPHIE KALEBA, University of Kent, UK HUMPHREY BURCHELL, University of Kent, UK STEFAN MARR, University of Kent, UK Thanks to partial evaluation and meta-tracing, it became practical to build language implementations that reach state-of-the-art peak performance by implementing only an interprete
Monitor AWS resources created by Terraform in Amazon DevOps Guru using tfdevops | Amazon Web Services
AWS DevOps & Developer Productivity Blog Monitor AWS resources created by Terraform in Amazon DevOps Guru using tfdevops This post was written in collaboration with Kapil Thangavelu, CTO at Stacklet Amazon DevOps Guru is a machine learning (ML) powered service that helps developers and operators automatically detect anomalies and improve application availability. DevOps Guru utilizes machine learn
Join a VS Code Dev Days event near you to learn about AI-assisted development in VS Code. Update 1.67.1: The update addresses this security issue. Update 1.67.2: The update addresses these issues. Downloads: Windows: x64 Arm64 | Mac: Universal Intel silicon | Linux: deb rpm tarball Arm snap Welcome to the April 2022 release of Visual Studio Code. There are many updates in this version that we hope
Parsing SQL - Strumenta
The code for this tutorial is on GitHub: parsing-sql SQL is a language to handle data in a relational database. If you worked with data you have probably worked with SQL. In this article we will talk about parsing SQL. It is in the same league of HTML: maybe you never learned it formally but you kind of know how to use it. That is great because if you know SQL, you know how to handle data. However
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
本番環境で採用すべき26のAWSセキュリティベストプラクティス
Why UUIDs won't protect your secrets
Building LLM applications for production
research!rsc: Coroutines for Go
Writing a C compiler in 500 lines of Python
Moving off of TypeScript
AI Agentのビジネス価値を計るバックテスト基盤の構築 - LayerX エンジニアブログ
How to find the AWS Account ID of any S3 Bucket | Tracebit
Choose Postgres queue technology
Run WebAssemblies in VS Code for the Web
Basic Feature Engineering with DuckDB
Go performance from version 1.2 to 1.18
How to create a Python package in 2022
A few words on Ruby's type annotations state
A Brief History of Ruby
Structural pattern matching in Python 3.10
RFC 9562: Universally Unique IDentifiers (UUIDs)
April 2022 (version 1.67)