Applications of transformer-based language models in bioinformatics: a survey

S Zhang, R Fan, Y Liu, S Chen, Q Liu… - Bioinformatics …, 2023 - academic.oup.com
The transformer-based language models, including vanilla transformer, BERT and GPT-3,
have achieved revolutionary breakthroughs in the field of natural language processing …

Transformer architecture and attention mechanisms in genome data analysis: a comprehensive review

SR Choi, M Lee - Biology, 2023 - mdpi.com
Simple Summary The rapidly advancing field of deep learning, specifically transformer-
based architectures and attention mechanisms, has found substantial applicability in …

The evolution, evolvability and engineering of gene regulatory DNA

ED Vaishnav, CG de Boer, J Molinet, M Yassour, L Fan… - Nature, 2022 - nature.com
Mutations in non-coding regulatory DNA sequences can alter gene expression, organismal
phenotype and fitness,–. Constructing complete fitness landscapes, in which DNA …

YOLO-LOGO: A transformer-based YOLO segmentation model for breast mass detection and segmentation in digital mammograms

Y Su, Q Liu, W Xie, P Hu - Computer Methods and Programs in …, 2022 - Elsevier
Background and objective Both mass detection and segmentation in digital mammograms
play a crucial role in early breast cancer detection and treatment. Furthermore, clinical …

CrystalGPT: Enhancing system-to-system transferability in crystallization prediction and control using time-series-transformers

N Sitapure, JSI Kwon - Computers & Chemical Engineering, 2023 - Elsevier
For prediction and real-time control tasks, machine-learning (ML)-based digital twins are
frequently employed. However, while these models are typically accurate, they are custom …

STGRNS: an interpretable transformer-based method for inferring gene regulatory networks from single-cell transcriptomic data

J Xu, A Zhang, F Liu, X Zhang - Bioinformatics, 2023 - academic.oup.com
Motivation Single-cell RNA-sequencing (scRNA-seq) technologies provide an opportunity to
infer cell-specific gene regulatory networks (GRNs), which is an important challenge in …

GENA-LM: a family of open-source foundational DNA language models for long sequences

V Fishman, Y Kuratov, A Shmelev, M Petrov, D Penzar… - bioRxiv, 2023 - biorxiv.org
Recent advancements in genomics, propelled by artificial intelligence, have unlocked
unprecedented capabilities in interpreting genomic sequences, mitigating the need for …

Artificial intelligence-assisted selection and efficacy prediction of antineoplastic strategies for precision cancer therapy

ZHE Zhang, X Wei - Seminars in Cancer Biology, 2023 - Elsevier
The rapid development of artificial intelligence (AI) technologies in the context of the vast
amount of collectable data obtained from high-throughput sequencing has led to an …

Transformers in healthcare: A survey

S Nerella, S Bandyopadhyay, J Zhang… - arXiv preprint arXiv …, 2023 - arxiv.org
With Artificial Intelligence (AI) increasingly permeating various aspects of society, including
healthcare, the adoption of the Transformers neural network architecture is rapidly changing …

Transformer technology in molecular science

J Jiang, L Ke, L Chen, B Dou, Y Zhu… - Wiley …, 2024 - Wiley Online Library
A transformer is the foundational architecture behind large language models designed to
handle sequential data by using mechanisms of self‐attention to weigh the importance of …