A review on the attention mechanism of deep learning
Attention has arguably become one of the most important concepts in the deep learning
field. It is inspired by the biological systems of humans that tend to focus on the distinctive …
field. It is inspired by the biological systems of humans that tend to focus on the distinctive …
Post-hoc interpretability for neural nlp: A survey
Neural networks for NLP are becoming increasingly complex and widespread, and there is a
growing concern if these models are responsible to use. Explaining models helps to address …
growing concern if these models are responsible to use. Explaining models helps to address …
Prottrans: Toward understanding the language of life through self-supervised learning
Computational biology and bioinformatics provide vast data gold-mines from protein
sequences, ideal for Language Models (LMs) taken from Natural Language Processing …
sequences, ideal for Language Models (LMs) taken from Natural Language Processing …
Quantifying attention flow in transformers
In the Transformer model," self-attention" combines information from attended embeddings
into the representation of the focal embedding in the next layer. Thus, across layers of the …
into the representation of the focal embedding in the next layer. Thus, across layers of the …
Ai alignment: A comprehensive survey
AI alignment aims to make AI systems behave in line with human intentions and values. As
AI systems grow more capable, the potential large-scale risks associated with misaligned AI …
AI systems grow more capable, the potential large-scale risks associated with misaligned AI …
Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?
A Jacovi, Y Goldberg - arXiv preprint arXiv:2004.03685, 2020 - arxiv.org
With the growing popularity of deep-learning based NLP models, comes a need for
interpretable systems. But what is interpretability, and what constitutes a high-quality …
interpretable systems. But what is interpretability, and what constitutes a high-quality …
Toward transparent ai: A survey on interpreting the inner structures of deep neural networks
The last decade of machine learning has seen drastic increases in scale and capabilities.
Deep neural networks (DNNs) are increasingly being deployed in the real world. However …
Deep neural networks (DNNs) are increasingly being deployed in the real world. However …
Explainable deep learning: A field guide for the uninitiated
Deep neural networks (DNNs) are an indispensable machine learning tool despite the
difficulty of diagnosing what aspects of a model's input drive its decisions. In countless real …
difficulty of diagnosing what aspects of a model's input drive its decisions. In countless real …
ERASER: A benchmark to evaluate rationalized NLP models
State-of-the-art models in NLP are now predominantly based on deep neural networks that
are opaque in terms of how they come to make predictions. This limitation has increased …
are opaque in terms of how they come to make predictions. This limitation has increased …
Attention mechanism in intelligent fault diagnosis of machinery: A review of technique and application
Attention Mechanism has become very popular in the field of mechanical fault diagnosis in
recent years and has become an important technique for scholars to study and apply. The …
recent years and has become an important technique for scholars to study and apply. The …