In today’s dynamic data management landscape, the terminology and concepts related to data storage and processing have become more...

In the last couple of days I’ve been hearing, reading and poking around the 1 Billion Row Challenge (1BRC) - a ”contest” for Java / JVM...

Inverted Indexes: why do you need one, and how to implement in Scala quickly and easily

Apache Spark has emerged as one of the most popular big data processing frameworks due to its speed, scalability, and ease of use....

This is a continuation of "Testing Spark Streaming locally with EmbeddedKafka". If you're not familar with EmbeddedKafka - I'd recommend...

It's been a while since my previous article in Spark/Scala series, where we ran Spark locally using Docker. And even before that we...

Performance reviews season being in full swing, got me thinking about role of managers in it, expectations from Individual Contributor's...

You have some kind of query - maybe it's written using Dataset API, maybe using Spark SQL. It reads from one or several Hive tables, or...

About a year ago I listened to Software Engineering Radio podcast episode with Alex Petrov, where Alex was discussing his new book,...

In previous post we've created a simple Spark app, and used Scalatest to check that it actually works. Even though we were creating a...

Years ago I wrote a blog post describing process of building and deploying a simple Spark app. This post is now too old to be of any use...