Deep neural-network-based language models (LMs) are increasingly applied to large-scale
protein sequence data to predict protein function. However, being largely black-box models …

Recent language models have shown impressive multilingual performance, even when not
explicitly trained for it. Despite this, there are concerns about the quality of their outputs …

Large language models (LLMs) have recently reached an impressive level of linguistic
capability, prompting comparisons with human language skills. However, there have been …

Subword tokenization has become the de-facto standard for tokenization, although
comparative evaluations of subword vocabulary quality across languages are scarce …

Cross-lingual semantic parsing transfers parsing capability from a high-resource language
(eg, English) to low-resource languages with scarce training data. Previous work has …

Previous work has demonstrated that pre-trained large language models (LLM) acquire
knowledge during pre-training which enables reasoning over relationships between words …

Word representation learning is a fundamental technique in cognitive computation that plays
a crucial role in enabling machines to understand and process human language. By …

When deriving contextualized word representations from language models, a decision
needs to be made on how to obtain one for out-of-vocabulary (OOV) words that are …

Tokenization is a critical part of modern NLP pipelines. However, contemporary tokenizers
for Large Language Models are based on statistical analysis of text corpora, without much …

Deep neural-network-based language models (LMs) are increasingly applied to large-scale
protein sequence data to predict protein function. However, being largely blackbox models …