Towards general and efficient online tuning for spark
The distributed data analytic system--Spark is a common choice for processing massive
volumes of heterogeneous data, while it is challenging to tune its parameters to achieve …
volumes of heterogeneous data, while it is challenging to tune its parameters to achieve …
Otterman: A novel approach of spark auto-tuning by a hybrid strategy
H Du, P Han, W Chen, Y Wang… - 2018 5th International …, 2018 - ieeexplore.ieee.org
Spark has become a very attractive platform for big data analytics in recent years due to its
unique advantages such as parallelism, fault tolerance, and complexity associated with …
unique advantages such as parallelism, fault tolerance, and complexity associated with …
Mjolnir: A framework agnostic auto-tuning system with deep reinforcement learning
Choosing the right setting for big data frameworks is an important yet difficult task. These
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
Mjolnir: A framework agnostic auto-tuning system with deep reinforcement learning
NB Slimane, H Sagaama, M Marwani… - Applied …, 2023 - search.proquest.com
Choosing the right setting for big data frameworks is an important yet difficult task. These
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
frameworks come with a complex set of parameters that need to be tuned to achieve the best …
Adaptive code learning for spark configuration tuning
Configuration tuning is vital to optimize the performance of big data analysis platforms like
Spark. Existing methods (eg auto-tuning relational databases) are not effective for tuning …
Spark. Existing methods (eg auto-tuning relational databases) are not effective for tuning …
Performance analysis and auto-tuning for spark in-memory analytics
D Nikitopoulou, D Masouros, S Xydis… - … Design, Automation & …, 2021 - ieeexplore.ieee.org
Recently the Apache Spark in-memory computing framework has gained a lot of attention,
due to its increased performance on large-scale data processing. Although Spark is highly …
due to its increased performance on large-scale data processing. Although Spark is highly …
Spark parameter tuning via trial-and-error
P Petridis, A Gounaris, J Torres - INNS Conference on Big Data, 2016 - Springer
Spark has been established as an attractive platform for big data analysis, since it manages
to hide most of the complexities related to parallelism, fault tolerance and cluster setting from …
to hide most of the complexities related to parallelism, fault tolerance and cluster setting from …
Pets: Bottleneck-aware spark tuning with parameter ensembles
Spark tuning with its dozens of parameters for performance improvement is both a challenge
and time consuming effort. Current techniques rely on trial-and-error or best guess utilizing …
and time consuming effort. Current techniques rely on trial-and-error or best guess utilizing …
Rover: An online Spark SQL tuning service via generalized transfer learning
Distributed data analytic engines like Spark are common choices to process massive data in
industry. However, the performance of Spark SQL highly depends on the choice of …
industry. However, the performance of Spark SQL highly depends on the choice of …
Tuning performance of Spark programs
H Zhang, Z Liu, L Wang - 2018 IEEE International Conference …, 2018 - ieeexplore.ieee.org
Along with the explosive growth of data, there is a great demand to speedup the ability to
process them. Although there are several platforms such as Spark that have made analysis …
process them. Although there are several platforms such as Spark that have made analysis …