Big Data

I have studied different shuffling mechanisms present in Spark and implemented a custom logger for collecting different parameters related to shuffling phase to understand their characteristics