The National Democratic Alliance (NDA) has emerged victorious in India’s 2024 general election, but with a smaller majority compared to 2019. According to post-election analysis by Goldman Sachs, JP Morgan,… The tech layoff wave is still going strong in 2024. Following significant workforce reductions in 2022 and 2023, this year has already seen 60,000 job cuts across 254 companies, according to i

One of the things I realised while doing research for my book is that contemporary software engineering still has a lot to learn from the 1970s. As we’re in such a fast-moving field, we often have a tendency of dismissing older ideas as irrelevant – and consequently, we end up having to learn the same lessons over and over again, the hard way. Although computers have got faster, data has got bigge

In an era where artificial intelligence (AI) is reshaping enterprises across the globe—be it in healthcare, finance, or manufacturing—it’s hard to overstate the transformation that AI has had on businesses, regardless of industry or size. At Cloudera, we recognize the urgent need for bold steps to harness this potential and dramatically accelerate the time to […] Read blog post

こんにちは。 Amazon Kinesisについて調べたり実装してみたりしたため、 モデルがよく似たApache Kafkaとの類似点や相違点が気になってきました。 というわけで、実際比べてみた結果どうだったのかをまとめてみます。 1.2つのプロダクトの類似点 Amazon KinesisとApache Kafkaの大きな類似点として、以下があります。 1.メッセージを取得したタイミングで削除するのではなく、一定期間経過後に削除するモデルを取っている Kestrel、RabbitMQといったプロダクトはメッセージを取得され、利用側から応答が返った段階で削除するモデルを取っている。 「メッセージキュー」と言いあらわした場合、削除するモデルの方がイメージ的には近い？ Amazon Kinesis、Apache Kafkaはメッセージの提供を行うのみで削除は行わない。一定時間経過後に削除している

Apache Kafka becoming the message bus to transfer huge volumes of data from various sources into Hadoop. It's also enabling many real-time system frameworks and use cases. Managing and building clients around Apache Kafka can be challenging. In this talk, we will go through the best practices in deploying Apache Kafka in production. How to Secure a Kafka Cluster, How to pick topic-partitions and u

We didn't use Kafka directly for two reasons: Migration cost is too high for us. We've already got large deployment of Suro clients and collectors before we needed Kafka. To switch to Kafka, we would have to complete the following tasks: Update Suro client to "swap in" Kafka producer while keeping full backward compatibility. No application should require any change except a rebuild and redeployme