Sample generation for testing Spark benchmarking using Hibench.
The spark parkameter used for spark generation was referenced from the currently published spark parameter tuning paper.
We reference this paper.
Xin, Jinhan, Kai Hwang, and Zhibin Yu. "LOCAT: Low-Overhead Online Configuration Auto-Tuning of Spark SQL Applications." Proceedings of the 2022 International Conference on Management of Data. 2022.
Paper Link : https://dl.acm.org/doi/abs/10.1145/3514221.3526157