Assessing the robustness of conversational agents using paraphrases

J Guichard, E Ruane, R Smith, D Bean… - 2019 IEEE …, 2019 - ieeexplore.ieee.org
J Guichard, E Ruane, R Smith, D Bean, A Ventresque
2019 IEEE international conference on artificial intelligence …, 2019ieeexplore.ieee.org
Assessing a conversational agent's understanding capabilities is critical, as poor user
interactions could seal the agent's fate at the very beginning of its lifecycle with users
abandoning the system. In this paper we explore the use of paraphrases as a testing tool for
conversational agents. Paraphrases, which are different ways of expressing the same intent,
are generated based on known working input by performing lexical substitutions. As the
expected outcome for this newly generated data is known, we can use it to assess the …
Assessing a conversational agent's understanding capabilities is critical, as poor user interactions could seal the agent's fate at the very beginning of its lifecycle with users abandoning the system. In this paper we explore the use of paraphrases as a testing tool for conversational agents. Paraphrases, which are different ways of expressing the same intent, are generated based on known working input by performing lexical substitutions. As the expected outcome for this newly generated data is known, we can use it to assess the agent's robustness to language variation and detect potential understanding weaknesses. As demonstrated by a case study, we obtain encouraging results as it appears that this approach can help anticipate potential understanding shortcomings and that these shortcomings can be addressed by the generated paraphrases.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果