Pets: Bottleneck-aware spark tuning with parameter ensembles
Spark tuning with its dozens of parameters for performance improvement is both a challenge
and time consuming effort. Current techniques rely on trial-and-error or best guess utilizing …
and time consuming effort. Current techniques rely on trial-and-error or best guess utilizing …
Mjolnir: A framework agnostic auto-tuning system with deep reinforcement learning
Choosing the right setting for big data frameworks is an important yet difficult task. These
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
Mjolnir: A framework agnostic auto-tuning system with deep reinforcement learning
NB Slimane, H Sagaama, M Marwani… - Applied …, 2023 - search.proquest.com
Choosing the right setting for big data frameworks is an important yet difficult task. These
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
Otterman: A novel approach of spark auto-tuning by a hybrid strategy
H Du, P Han, W Chen, Y Wang… - 2018 5th International …, 2018 - ieeexplore.ieee.org
Spark has become a very attractive platform for big data analytics in recent years due to its
unique advantages such as parallelism, fault tolerance, and complexity associated with …
unique advantages such as parallelism, fault tolerance, and complexity associated with …
Spark parameter tuning via trial-and-error
P Petridis, A Gounaris, J Torres - INNS Conference on Big Data, 2016 - Springer
Spark has been established as an attractive platform for big data analysis, since it manages
to hide most of the complexities related to parallelism, fault tolerance and cluster setting from …
to hide most of the complexities related to parallelism, fault tolerance and cluster setting from …
Algorithmic Proficiency in Spark Configuration Tuning: An Empirical Study using Execution Time Metrics across Varied Workloads
In the realm of big data, where datasets of immense scale pose processing challenges,
distributed processing platforms like open-source Apache Spark have emerged to address …
distributed processing platforms like open-source Apache Spark have emerged to address …
Tuning performance of Spark programs
H Zhang, Z Liu, L Wang - 2018 IEEE International Conference …, 2018 - ieeexplore.ieee.org
Along with the explosive growth of data, there is a great demand to speedup the ability to
process them. Although there are several platforms such as Spark that have made analysis …
process them. Although there are several platforms such as Spark that have made analysis …
An experimental comparison between genetic algorithm and particle swarm optimization in spark performance tuning
Y Wang, Q Liu, JQ Yu, Z Yu - Proceedings of the first Workshop on …, 2017 - dl.acm.org
The most popular in-memory computing framework---Spark---has a number of performance-
critical configuration parameters. Manually tuning these parameters for optimized …
critical configuration parameters. Manually tuning these parameters for optimized …
Towards automatic tuning of apache spark configuration
Apache Spark provides a large number of configuration settings that may be tuned to
improve the performance of specific applications running on the platform. However, it is non …
improve the performance of specific applications running on the platform. However, it is non …
You only run once: spark auto-tuning from a single run
DB Prats, FA Portella, CHA Costa… - IEEE Transactions on …, 2020 - ieeexplore.ieee.org
Tuning configurations of Spark jobs is not a trivial task. State-of-the-art auto-tuning systems
are based on iteratively running workloads with different configurations. During the …
are based on iteratively running workloads with different configurations. During the …