Tuning performance of Spark programs

H Zhang, Z Liu, L Wang - 2018 IEEE International Conference …, 2018 - ieeexplore.ieee.org
Along with the explosive growth of data, there is a great demand to speedup the ability to
process them. Although there are several platforms such as Spark that have made analysis …

Towards automatic tuning of apache spark configuration

N Nguyen, MMH Khan, K Wang - 2018 IEEE 11th International …, 2018 - ieeexplore.ieee.org
Apache Spark provides a large number of configuration settings that may be tuned to
improve the performance of specific applications running on the platform. However, it is non …

Otterman: A novel approach of spark auto-tuning by a hybrid strategy

H Du, P Han, W Chen, Y Wang… - 2018 5th International …, 2018 - ieeexplore.ieee.org
Spark has become a very attractive platform for big data analytics in recent years due to its
unique advantages such as parallelism, fault tolerance, and complexity associated with …

SparkOT: Diagnosing operation level inefficiency in spark

H Zhou, Y Li, J Jia, W Qi, H Yang - 2018 IEEE 20th …, 2018 - ieeexplore.ieee.org
Spark is a popular big data framework aiming to speed up computation by caching data in
memory. Spark splits the execution of an application into multiple operations which form a …

Insights on apache spark usage by mining stack overflow questions

LJ Rodríguez, X Wang, J Kuang - 2018 IEEE International …, 2018 - ieeexplore.ieee.org
Apache Spark is one of the most popular big data tools. Despite its popularity, there are no
studies regarding its overall usage among software developers. As such, essential …

Sparker: Optimizing spark for heterogeneous clusters

N Garg, D Janakiram - 2018 IEEE International Conference on …, 2018 - ieeexplore.ieee.org
Spark is an in-memory big data analytics framework which has replaced Hadoop as the de
facto standard for processing big data in cloud platforms. These frameworks run on cloud …

Understanding the influence of configuration settings: An execution model-driven framework for apache spark platform

N Nguyen, MMH Khan, Y Albayram… - 2017 IEEE 10th …, 2017 - ieeexplore.ieee.org
Apache Spark provides numerous configuration settings that can be tuned to improve the
performance of specific applications running on the platform. However, due to its multi-stage …

SPOAHA: spark program optimizer based on artificial hummingbird algorithm

M Wang, J Zhen, Y Ma, X Huang, H Zhang - International Conference on …, 2023 - Springer
In this era of the Internet of Things (IoT), a large number of sensor devices collect and
generate various sensing data over time. It is very essential to mine fresh information by …

Pets: Bottleneck-aware spark tuning with parameter ensembles

TBG Perez, W Chen, R Ji, L Liu… - 2018 27th International …, 2018 - ieeexplore.ieee.org
Spark tuning with its dozens of parameters for performance improvement is both a challenge
and time consuming effort. Current techniques rely on trial-and-error or best guess utilizing …

Dione: Profiling spark applications exploiting graph similarity

N Zacheilas, S Maroulis… - 2017 IEEE International …, 2017 - ieeexplore.ieee.org
In recent years distributed processing frameworks such as Apache Spark have been utilized
for running big data applications. Predicting the application's execution time has been an …