Measure and improve robustness in NLP models: A survey

X Wang, H Wang, D Yang - arXiv preprint arXiv:2112.08313, 2021 - arxiv.org
As NLP models achieved state-of-the-art performances over benchmarks and gained wide
applications, it has been increasingly important to ensure the safe deployment of these …

On robustness of prompt-based semantic parsing with large pre-trained language model: An empirical study on codex

TY Zhuo, Z Li, Y Huang, F Shiri, W Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
Semantic parsing is a technique aimed at constructing a structured representation of the
meaning of a natural-language question. Recent advancements in few-shot language …

Towards robustness of text-to-SQL models against natural and realistic adversarial table perturbation

X Pi, B Wang, Y Gao, J Guo, Z Li, JG Lou - arXiv preprint arXiv:2212.09994, 2022 - arxiv.org
The robustness of Text-to-SQL parsers against adversarial perturbations plays a crucial role
in delivering highly reliable applications. Previous studies along this line primarily focused …

Compositional generalization for multi-label text classification: A data-augmentation approach

Y Chai, Z Li, J Liu, L Chen, F Li, D Ji… - Proceedings of the AAAI …, 2024 - ojs.aaai.org
Despite significant advancements in multi-label text classification, the ability of existing
models to generalize to novel and seldom-encountered complex concepts, which are …

Factual: A benchmark for faithful and consistent textual scene graph parsing

Z Li, Y Chai, TY Zhuo, L Qu, G Haffari, F Li, D Ji… - arXiv preprint arXiv …, 2023 - arxiv.org
Textual scene graph parsing has become increasingly important in various vision-language
applications, including image caption evaluation and image retrieval. However, existing …

The best of both worlds: Combining human and machine translations for multilingual semantic parsing with active learning

Z Li, L Qu, PR Cohen, RV Tumuluri… - arXiv preprint arXiv …, 2023 - arxiv.org
Multilingual semantic parsing aims to leverage the knowledge from the high-resource
languages to improve low-resource semantic parsing, yet commonly suffers from the data …

Energy-latency attacks to on-device neural networks via sponge poisoning

Z Wang, S Huang, Y Huang, H Cui - … of the 2023 Secure and Trustworthy …, 2023 - dl.acm.org
In recent years, on-device deep learning has gained attention as a means of developing
affordable deep learning applications for mobile devices. However, on-device models are …

Dr. spider: A diagnostic evaluation benchmark towards text-to-sql robustness

S Chang, J Wang, M Dong, L Pan, H Zhu, AH Li… - arXiv preprint arXiv …, 2023 - arxiv.org
Neural text-to-SQL models have achieved remarkable performance in translating natural
language questions into SQL queries. However, recent studies reveal that text-to-SQL …

Improving Generalization in Semantic Parsing by Increasing Natural Language Variation

I Saparina, M Lapata - arXiv preprint arXiv:2402.08666, 2024 - arxiv.org
Text-to-SQL semantic parsing has made significant progress in recent years, with various
models demonstrating impressive performance on the challenging Spider benchmark …

Enhancing Robustness of AI Offensive Code Generators via Data Augmentation

C Improta, P Liguori, R Natella, B Cukic… - arXiv preprint arXiv …, 2023 - arxiv.org
In this work, we present a method to add perturbations to the code descriptions to create
new inputs in natural language (NL) from well-intentioned developers that diverge from the …