January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience with neural networks. This implementation is for educational purposes, so it's missing lots of features/improv
This all started with a blog post back in 2020, from a tech curiosity: what's the fastest way to scale containers on AWS? Is ECS faster than EKS? What about Fargate? Is there a difference between ECS on Fargate and EKS on Fargate? I had to know this to build better architectures for my clients. In 2021, containers got even better, and I was lucky enough to get a preview and present just how fast t
There are lots of posts trying to show how simple it is to get started with Kubernetes. But many of these posts use complicated Kubernetes jargon for that, so even those with some prior server-side knowledge might be bewildered. Let me try something different here. Instead of explaining one unfamiliar matter (how to run a web service in Kubernetes?) with another (you just need a manifest, with thr
AWS Lambda standardizes billing for INIT Phase | Amazon Web Services
AWS Compute Blog AWS Lambda standardizes billing for INIT Phase Effective August 1, 2025, AWS will standardize billing for the initialization (INIT) phase across all AWS Lambda function configurations. This change specifically affects on-demand invocations of Lambda functions packaged as ZIP files that use managed runtimes, for which the INIT phase duration was previously unbilled. This update sta
Serverless development and feedback loopsWith Serverless Dagster Cloud you can develop and deploy Dagster code without setting up either a local development environment or any cloud infrastructure. When you commit a change to GitHub, a GitHub Action builds and deploys your code directly to Dagster Cloud, where you can view and interact with your Dagster objects in the UI. With Dagster Cloud, remo
AWS LambdaにおけるJava実装時のSpringとMicronautの比較 - ZOZO TECH BLOG
はじめに ブランドソリューション開発部プロダクト開発チームの木目沢です。 Fulfillment by ZOZO（以下、FBZ）で提供しているAPIの開発に携わっています。以前「FBZにおけるドメイン駆動設計（DDD）とサーバーレスアーキテクチャを組み合わせた設計戦術」という記事を公開しました。そこでは、AWS Lambdaを中心としたサーバーレスアーキテクチャを採用していること、ドメイン駆動設計でAWSのサービス処理とビジネス処理を分離していることをご紹介しました。 FBZはリリース前の設定時にはJavaも検討していました。しかし、結果として採用を見送ることにしました。その理由とリリースから4年が経過した今、改めてJavaに関して調査した結果を本記事ではご紹介します。 JavaではなくPythonを選択した理由 FBZの設計をしていた当時、Lambdaで使用可能な言語は、Node.js、
statesunsetinwritingdate6/12/2020🌇 Sunset The sun is setting on these articles, they are still useful, but they are not the future. They are the past, and likely outdated. Read with caution. Writing code and deploying it to AWS Lambda is as easy as baking a cake (depending on the type of cake). Lambda performs the heavy lifting for you, from provisioning to scaling. But where is the magic happeni
Using AWS CodePipeline for deploying container images to AWS Lambda Functions | Amazon Web Services
AWS DevOps & Developer Productivity Blog Using AWS CodePipeline for deploying container images to AWS Lambda Functions AWS Lambda launched support for packaging and deploying functions as container images at re:Invent 2020. In the post working with Lambda layers and extensions in container images, we demonstrated packaging Lambda Functions with layers while using container images. This post will t
はじめての自然言語処理 ELECTRA（BERT の事前学習手法の改良）による固有表現抽出の検証 | オブジェクトの広場
今回は BERT における事前学習の改良手法である ELECTRA の検証です。ELECTRA はモデルサイズ、データ、計算量が同一条件であればオリジナルの BERT を凌ぐ性能とのことなので結果が楽しみなところです。事前学習をした後のファインチューニングは、いつも livedoor News Corpus の文書分類ばかりだったので、今回は固有表現抽出を試すことにしました。 1. はじめに 今回は BERT における事前学習の改良手法である ELECTRA 1 の検証です。 BERT に関しては 第3回 で取り上げていますが、トークン化が Sentencepiece である為、トークン単位での処理に難がありました2。今回は ELECTRA を試すにあたり、そのあたりの対応も入れ、 Megagon Labs さんから公開されている UD_Japanese-GSD v2.6-NE 3 を使っ
Create a modern data platform using the Data Build Tool (dbt) in the AWS Cloud | Amazon Web Services
AWS Big Data Blog Create a modern data platform using the Data Build Tool (dbt) in the AWS Cloud Building a data platform involves various approaches, each with its unique blend of complexities and solutions. A modern data platform entails maintaining data across multiple layers, targeting diverse platform capabilities like high performance, ease of development, cost-effectiveness, and DataOps fea
CloudFormationのテンプレートでよく使う組み込み関数に!Refと!Subがあります。 これらの違いを整理します。 基本的な使い方 基本的な作用 使い方 誤った使い方 !Subの問題点 文字列との結合 !Subの場合 !Refの場合 結論 基本的な使い方 基本的な作用 基本的な作用は同一で、論理名を参照し、規定の値に変換します 変換可能な値は、!Refの公式ドキュメントで参照できる他、各リソースの「戻り値」の項目で確認できる(例えば、Lambdaの公式ドキュメント。 なお、ドキュメントでは!Refの戻り値しか紹介されていませんが、!Subを使っても全く同じ値が帰ってきます。 使い方 値を参照するに当たり、論理名を記載します。各関数は、リソースのタイプごとに定められた値を参照し、置換します。 使い方の注意点として、!Refは論理名を直接表記するのに対し、!Subは論理名を${}でく
Understanding and Remediating Cold Starts: An AWS Lambda Perspective | Amazon Web Services
AWS Compute Blog Understanding and Remediating Cold Starts: An AWS Lambda Perspective Cold starts are an important consideration when building applications on serverless platforms. In AWS Lambda, they refer to the initialization steps that occur when a function is invoked after a period of inactivity or during rapid scale-up. While typically brief and infrequent, cold starts can introduce addition
v2022: Austin Huang, Suraj Subramanian, Jonathan Sum, Khalid Almubarak, and Stella Biderman. Original: Sasha Rush. The Transformer has been on a lot of people’s minds over the last year five years. This post presents an annotated version of the paper in the form of a line-by-line implementation. It reorders and deletes some sections from the original paper and adds comments throughout. This docume
MLOps foundation roadmap for enterprises with Amazon SageMaker | Amazon Web Services
Artificial Intelligence MLOps foundation roadmap for enterprises with Amazon SageMaker As enterprise businesses embrace machine learning (ML) across their organizations, manual workflows for building, training, and deploying ML models tend to become bottlenecks to innovation. To overcome this, enterprises needs to shape a clear operating model defining how multiple personas, such as data scientist
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
GPT in 60 Lines of NumPy | Jay Mody
Scaling containers on AWS in 2022
How Kubernetes Reinvented Virtual Machines (in a good sense)
Dagster Cloud: 5X Faster Deployments
Behind the Scenes Lambda
【CloudFormation】!Subと!Refの使い分け - Tycoh’s blog
The Annotated Transformer