A Foray into Nested Data using Spark SQL

Published in 2021

Nested data is often our least favorite type of data since it can be challenging and time-consuming to wrangle. In this workshop, we distill foundational concepts about two common types of nested columns: arrays and structs. Using SQL Analytics, we will be exploring a dataset curated by the Manhattan Chocolate Society on dark chocolate with varying flavors, ratings, and places of origin. You will learn how to create nested data from flat data, manipulate and conduct aggregate calculations on nested columns; and also perhaps more importantly, about chocolate!

Collaboration with Fred Abood.