Estimating the semantic similarity between text data is one of the challenging and open
research problems in the field of Natural Language Processing (NLP). The versatility of …

Pre-trained language models (PLMs) have been the de facto paradigm for most natural
language processing tasks. This also benefits the biomedical domain: researchers from …

Language model (LM) pretraining can learn various knowledge from text corpora, helping
downstream tasks. However, existing methods such as BERT model a single document, and …

Pretraining large neural language models, such as BERT, has led to impressive gains on
many natural language processing (NLP) tasks. However, most pretraining efforts focus on …

Inspired by the success of the General Language Understanding Evaluation benchmark, we
introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to …

Motivation Information retrieval (IR) is essential in biomedical knowledge acquisition and
clinical decision support. While recent progress has shown that language model encoders …

Electronic health records (EHRs), digital collections of patient healthcare events and
observations, are ubiquitous in medicine and critical to healthcare delivery, operations, and …

Large language models (LLMs) have made significant progress in various domains,
including healthcare. However, the specialized nature of clinical language understanding …

Transformer-based pretrained language models (PLMs) have started a new era in modern
natural language processing (NLP). These models combine the power of transformers …

Large neural language models have transformed modern natural language processing
(NLP) applications. However, fine-tuning such models for specific tasks remains challenging …