Tutorial: Spark application architecture and clusters

Before you begin your journey as an Apache Spark programmer, you should have a solid understanding of the Spark application architecture and how applications are executed on a Spark cluster. This article closely examines the components of a Spark application, looks at how these components work together, and looks at how Spark applications run on… Continue reading Tutorial: Spark application architecture and clusters

Fix your data before you move it to the cloud

Data seems to be a forgotten component of a good system architecture. In cloud migrations, I’m often taken back how the data that was bad on-premises is worse in the cloud. What does bad data look like? Poor data structures, redundant data, no single source of truth, and security and compliance issues all top the… Continue reading Fix your data before you move it to the cloud