Apache: Big Data North America 2017

Hitachi has focused on social innovation business. It has constantly evolved to create sustainable business products and solutions to enhance the quality of life across the globe. Now we are leveraging smart meter data for electric utilities. To meet their needs, we compared the performance of batch processing for aggregating data from smart meters using Hadoop (MapReduce) and Spark 1.6 and Spark 2.0 with changing some parameters (the amount of input data, the logic of processing, input file format, etc.).In this session, we report the results of performance test above.