[PDF][PDF] Classifying weather images using deep neural networks for large scale datasets

S Mittal, OP Sangwan - … Journal of Advanced Computer Science and …, 2023 - academia.edu
Classifying weather from outdoor images helps prevent road accidents, schedule outdoor
activities, and improve the reliability of vehicle assistant driving and outdoor video …

Machine-learning based memory prediction model for data parallel workloads in apache spark

R Myung, S Choi - Symmetry, 2021 - mdpi.com
A lack of memory can lead to job failures or increase processing times for garbage
collection. However, if too much memory is provided, the processing time is only marginally …

[HTML][HTML] 一种Spark 作业配置参数智能优化方法

阮树骅, 潘梵梵, 陈兴蜀, 罗永刚, 吴天雄 - 2020 - jsuese.cnjournals.com
Spark 的配置参数对作业运行性能有较大影响, 针对配置参数种类多, 参数搜索空间大,
参数间相互影响导致人工配置参数调优效率低下的问题, 提出了一种Spark …

Parameter optimization of Spark in heterogeneous environment based on hyperband

D Chang, Z Qiao, L Li, Q Zheng - 2021 2nd International …, 2021 - ieeexplore.ieee.org
There are many default parameters in Spark, which have a great influence on job
performance. Different jobs require different parameter combos, resulting in the need to tune …

Parameter optimization on spark for particulate matter estimation

Z Yu, Z Wang, L Bai, L Chen, J Tao - 2021 Workshop on Algorithm and …, 2021 - dl.acm.org
With the rapid growth of remote sensing satellites, the volume of remote sensing data has
been continuously increasing, which makes it necessary to utilize the big data platform for …

Performance Evaluation of Machine Learning Models on Apache Spark: An Empirical Study

AZ Yamani, SJ Alsunaidi… - 2022 14th International …, 2022 - ieeexplore.ieee.org
Artificial intelligence (AI) and machine learning significantly improve many sectors, such as
education, healthcare, and industry. Machine learning techniques mainly depend on the …

Implementing query operations for Knowledge Graph OLAP in Apache Spark/Author Jennifer Klar, BSc

J Klar - 2023 - epub.jku.at
Knowledge Graph OLAP kombiniert das Konzept von Knowledge Graphs (KG) mit einer
multidimensionalen Sicht auf Daten, wie sie im Bereich des Online Analytical Processing …

Spark Performance Optimization Analysis with Multi-Layer Parameter using Shuffling and Scheduling in Different Data Caching Options

DM Adinew, Z Shijie, AT Alemu - 2021 16th International …, 2021 - ieeexplore.ieee.org
As social networking services and e-commerce are growing rapidly, the number of online
users also dynamically growing which facilitates the contribution of huge content to the …

Spark Performance Optimization Analysis With Multi-Layer Parameter Using Shuffling and Scheduling With Data Serialization in Different Data Caching Options

M Deleli, DM Adinew, AT Alemu - Journal of Technological …, 2021 - igi-global.com
As social networking services and e-commerce are growing rapidly, the number of online
users also dynamically growing that facilitate contribution of huge contents to digital world. In …

lpt: a Tool for Tuning the Level of Parallelism of Spark Applications

E Rosales, A Rosa, W Binder - 2018 25th Asia-Pacific Software …, 2018 - ieeexplore.ieee.org
Spark is increasingly becoming the platform of choice for several big-data analyses mainly
due to its fast, fault-tolerant, and in-memory processing model. Despite the popularity and …