Transformer-based models have pushed state of the art in many areas of NLP, but our
understanding of what is behind their success is still limited. This paper is the first survey of …

Abstract We introduce Transformer Grammars (TGs), a novel class of Transformer language
models that combine (i) the expressive power, scalability, and strong performance of …

Acceptance testing is crucial to determine whether a system fulfills end-user requirements.
However, the creation of acceptance tests is a laborious task entailing two major …

Dependency-based models are widely used to extract semantic relations in text. Most
existing dependency-based models establish stacked structures to merge contextual and …

We propose a novel Transformer encoder-based architecture with syntactical knowledge
encoded for intent detection and slot filling. Specifically, we encode syntactic knowledge into …

In the modern world human-robot relations, language plays a significant role. One used to
view language as a purely human technology, but today language is being mastered by non …

[Objective] This paper analyzes and summarizes the main optimization methods of the BERT
language representation model released by Google to provide reference for future studies …

Pre-trained transformer-based models, such as BERT, have shown excellent performance in
most natural language processing benchmark tests, but we still lack a good understanding …

Causal relations in natural language (NL) requirements convey strong, semantic
information. Automatically extracting such causal information enables multiple use cases …

Textual representation learners trained on large amounts of data have achieved notable
success on downstream tasks; intriguingly, they have also performed well on challenging …