Transformer-based deep learning for predicting protein properties in the life sciences
Recent developments in deep learning, coupled with an increasing number of sequenced
proteins, have led to a breakthrough in life science applications, in particular in protein …
proteins, have led to a breakthrough in life science applications, in particular in protein …
[HTML][HTML] AMMU: a survey of transformer-based biomedical pretrained language models
KS Kalyan, A Rajasekharan, S Sangeetha - Journal of biomedical …, 2022 - Elsevier
Transformer-based pretrained language models (PLMs) have started a new era in modern
natural language processing (NLP). These models combine the power of transformers …
natural language processing (NLP). These models combine the power of transformers …
Transfer learning in biomedical natural language processing: an evaluation of BERT and ELMo on ten benchmarking datasets
Inspired by the success of the General Language Understanding Evaluation benchmark, we
introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to …
introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to …
Clinicalbert: Modeling clinical notes and predicting hospital readmission
Clinical notes contain information about patients that goes beyond structured data like lab
values and medications. However, clinical notes have been underused relative to structured …
values and medications. However, clinical notes have been underused relative to structured …
Self-alignment pretraining for biomedical entity representations
Despite the widespread success of self-supervised learning via masked language models
(MLM), accurately capturing fine-grained semantic relationships in the biomedical domain …
(MLM), accurately capturing fine-grained semantic relationships in the biomedical domain …
BioWordVec, improving biomedical word embeddings with subword information and MeSH
Distributed word representations have become an essential foundation for biomedical
natural language processing (BioNLP), text mining and information retrieval. Word …
natural language processing (BioNLP), text mining and information retrieval. Word …
Pretrained language models for biomedical and clinical tasks: understanding and extending the state-of-the-art
A large array of pretrained models are available to the biomedical NLP (BioNLP) community.
Finding the best model for a particular task can be difficult and time-consuming. For many …
Finding the best model for a particular task can be difficult and time-consuming. For many …
Who do we invent for? Patents by women focus more on women's health, but few women get to invent
Women engage in less commercial patenting and invention than do men, which may affect
what is invented. Using text analysis of all US biomedical patents filed from 1976 through …
what is invented. Using text analysis of all US biomedical patents filed from 1976 through …
Enhancing clinical concept extraction with contextual embeddings
Objective Neural network–based representations (“embeddings”) have dramatically
advanced natural language processing (NLP) tasks, including clinical NLP tasks such as …
advanced natural language processing (NLP) tasks, including clinical NLP tasks such as …
COVID-19 sentiment analysis via deep learning during the rise of novel cases
Social scientists and psychologists take interest in understanding how people express
emotions and sentiments when dealing with catastrophic events such as natural disasters …
emotions and sentiments when dealing with catastrophic events such as natural disasters …