Vision: Evaluating scenario suitableness for DNN models by mirror synthesis

Z Chen, H Wang, C Xu, X Ma… - 2019 26th Asia-Pacific …, 2019 - ieeexplore.ieee.org
Z Chen, H Wang, C Xu, X Ma, C Cao
2019 26th Asia-Pacific Software Engineering Conference (APSEC), 2019ieeexplore.ieee.org
Software systems assisted with deep neural networks (DNNs) are gaining increasing
popularities. However, one outstanding problem is to judge whether a given application
scenario suits a DNN model, whose answer highly affects its concerned system's
performance. Existing work indirectly addressed this problem by seeking for higher test
coverage or generating adversarial inputs. One pioneering work is SynEva, which exactly
addressed this problem by synthesizing mirror programs for scenario suitableness …
Software systems assisted with deep neural networks (DNNs) are gaining increasing popularities. However, one outstanding problem is to judge whether a given application scenario suits a DNN model, whose answer highly affects its concerned system's performance. Existing work indirectly addressed this problem by seeking for higher test coverage or generating adversarial inputs. One pioneering work is SynEva, which exactly addressed this problem by synthesizing mirror programs for scenario suitableness evaluation of general machine learning programs, but fell short in supporting DNN models. In this paper, we propose VISION to eValuatIng Scenario suItableness fOr DNN models, specially catered for DNN characteristics. We conducted experiments on a real-world self-driving dataset Udacity, and the results show that VISION was effective in evaluating scenario suitableness for DNN models with an accuracy of 75.6-89.0% as compared to that of SynEva, 50.0-81.8%. We also explored different meta-models in VISION, and found out that the decision tree logic learner meta-model could be the best one for balancing VISION's effectiveness and efficiency.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果