Runtime management of data quality for scientific observatories using edge and in-transit resources
AR Zamani, D Balouek-Thomert… - 2018 30th …, 2018 - ieeexplore.ieee.org
2018 30th International Symposium on Computer Architecture and …, 2018•ieeexplore.ieee.org
Modern Cyberinfrastructures (CIs) operate to bring content produced from remote data
sources such as sensors and scientific instruments and deliver it to end users and workflow
applications. Maintaining data quality/resolution and on-time data delivery while considering
an increasing number of computing, storage and network resources requires a reactive
system, able to adapt to changing demands. In this paper, we propose a modelization of
such system by expressing the dynamic stage of resources in the context of edge and in …
sources such as sensors and scientific instruments and deliver it to end users and workflow
applications. Maintaining data quality/resolution and on-time data delivery while considering
an increasing number of computing, storage and network resources requires a reactive
system, able to adapt to changing demands. In this paper, we propose a modelization of
such system by expressing the dynamic stage of resources in the context of edge and in …
Modern Cyberinfrastructures (CIs) operate to bring content produced from remote data sources such as sensors and scientific instruments and deliver it to end users and workflow applications. Maintaining data quality/resolution and on-time data delivery while considering an increasing number of computing, storage and network resources requires a reactive system, able to adapt to changing demands. In this paper, we propose a modelization of such system by expressing the dynamic stage of resources in the context of edge and in-transit computing. By considering resource utilization, approximation techniques and users' constraints, our proposed engine is generating mappings of workflow stages on heterogeneous geo-distributed resources. We specifically propose a runtime management layer that adapts the data resolution being delivered to the users by implementing feedback loops over the resources involved in the delivery and processing of the data streams. We implement our model into a subscription-based data streaming framework which enables integration of large facilities and advanced CIs. Experimental results show that dynamically adapting data resolution can overcome bandwidth limitation in wide area streaming analytics.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果