はてなブックマーク

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook Why new Macs break your Docker build, and how to fix it by Itamar Turner-Trauring Last updated 03 May 2024, originally created 20 Jun 2022 One of the promises of Docker is reproducibility: you can build an image on a different machine, and assuming you’ve

Processing large JSON files in Python without running out of memory by Itamar Turner-Trauring Last updated 06 Jan 2023, originally created 14 Mar 2022 If you need to process a large JSON file in Python, it’s very easy to run out of memory. Even if the raw data fits in memory, the Python representation can increase memory usage even more. And that means either slow processing, as your program swaps

Faster Python calculations with Numba: 2 lines of code, 13× speed-up by Itamar Turner-Trauring Last updated 06 Jan 2023, originally created 11 Feb 2022 Python is a slow language, so computation is best delegated to code written in something faster. You can do this with existing libraries like NumPy and SciPy, but what happens when you need to implement a new algorithm, and you don’t want to write

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook The worst so-called “best practice” for Docker by Itamar Turner-Trauring Last updated 01 Oct 2021, originally created 23 Mar 2021 Somebody is always wrong on the Internet, and bad Docker packaging advice is quite common. But one particular piece of advice

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook A deep dive into the “official” Docker image for Python by Itamar Turner-Trauring Last updated 20 Jan 2022, originally created 19 Aug 2020 The “official” Python image for Docker is quite popular, and in fact I recommend one of its variations as a base ima

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook “Let’s use Kubernetes!” Now you have 8 problems by Itamar Turner-Trauring Last updated 01 Oct 2021, originally created 04 Mar 2020 If you’re using Docker, the next natural step seems to be Kubernetes, aka K8s: that’s how you run things in production, righ

Create production-ready Docker images in just one hour You’re packaging your Python application with Docker. It’s going to be running in production. And now you have a whole new set of worries: Insecure images will put your production data at risk. Slow CI builds will bottleneck your whole team, wasting expensive developer time. Silent crashes become that much harder to debug, wasting even more de

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook Using Alpine can make Python Docker builds 50× slower by Itamar Turner-Trauring Last updated 21 Mar 2023, originally created 29 Jan 2020 When you’re choosing a base image for your Docker image, Alpine Linux is often recommended. Using Alpine, you’re told,

Why your multiprocessing Pool is stuck (it’s full of sharks!) by Itamar Turner-Trauring Last updated 17 Feb 2023, originally created 04 Sep 2018 You’re using multiprocessing to run some code across multiple processes, and it just—sits there. It’s stuck. You check CPU usage—nothing happening, it’s not doing any work. What’s going on? In many cases you can fix this with a single line of code—skip to

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook The best Docker base image for your Python application (May 2024) by Itamar Turner-Trauring Last updated 13 Sep 2024, originally created 30 Aug 2021 When you’re building a Docker image for your Python application, you’re building on top of an existing ima

When your data doesn’t fit in memory: the basic techniques by Itamar Turner-Trauring Last updated 06 Jan 2023, originally created 08 Nov 2019 You’re writing software that processes data, and it works fine when you test it on a small sample file. But when you load the real data, your program crashes. The problem is that you don’t have enough memory—if you have 16GB of RAM, you can’t load a 100GB fi

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook Faster Docker builds with pipenv, poetry, or pip-tools by Itamar Turner-Trauring Last updated 21 Mar 2023, originally created 17 Jun 2019 Docker builds can be slow, and waiting for a build to finish is probably not how you want to spend your time. So you

Introduction to Dockerizing for Production Improve your DevOps skills: learn an iterative process for Dockerizing your code. Get your free ebook Broken by default: why you should avoid most Dockerfile examples by Itamar Turner-Trauring Last updated 13 Sep 2024, originally created 28 May 2019 When it’s time to package up your Python application into a Docker image, the natural thing to do is search

Realistic, easy, and fast enough: database tests with Docker by Itamar Turner-Trauring Last updated 22 Dec 2021, originally created 09 Feb 2019 You’re writing an application that talks to a database—PostgreSQL, MySQL, MongoDB—and you need to write some tests. You want three things out of your tests: Realism: the closer your test setup is to production, the more likely you are to catch bugs with te