Experiences with the model-based generation of big data pipelines

H Eichelberger, C Qin, K Schmid - 2017 - dl.gi.de
2017dl.gi.de
Developing Big Data applications implies a lot of schematic or complex structural tasks,
which can easily lead to implementation errors and incorrect analysis results. In this paper,
we present a model-based approach that supports the automatic generation of code to
handle these repetitive tasks, enabling data engineers to focus on the functional aspects
without being distracted by technical issues. In order to identify a solution, we analyzed
different Big Data stream-processing frameworks, extracted a common graph-based model …
Abstract
Developing Big Data applications implies a lot of schematic or complex structural tasks, which can easily lead to implementation errors and incorrect analysis results. In this paper, we present a model-based approach that supports the automatic generation of code to handle these repetitive tasks, enabling data engineers to focus on the functional aspects without being distracted by technical issues. In order to identify a solution, we analyzed different Big Data stream-processing frameworks, extracted a common graph-based model for Big Data streaming applications and developed a tool to graphically design and generate such applications in a model-based fashion (in this work for Apache Storm). Here, we discuss the concepts of the approach, the tooling and, in particular, experiences with the approach based on feedback of our partners.
dl.gi.de
以上显示的是最相近的搜索结果。 查看全部搜索结果