Applying Reinforcement Learning (RL) to sequence generation models enables the direct
optimization of long-term rewards (\textit {eg,} BLEU and human feedback), but typically …

Transformers have dominated empirical machine learning models of natural language
processing. In this paper, we introduce basic concepts of Transformers and present key …

Learning multiscale Transformer models has been evidenced as a viable approach to
augmenting machine translation systems. Prior research has primarily focused on treating …

Large language models achieve state-of-the-art performance on sequence generation
evaluation, but typically have a large number of parameters. This is a computational …

Although neural machine translation (NMT) models perform well in the general domain, it
remains rather challenging to control their generation behavior to satisfy the requirement of …

Long sequence time-series forecasting (LSTF) is a significant and challenging task. Many
real-world applications require long-term forecasting of time series. In recent years …

Conventional neural machine translation (NMT) models typically use subwords and words
as the basic units for model input and comprehension. However, complete words and …

Through end-to-end training to predict the next token, LLMs have become valuable tools for
various tasks. Enhancing their core training in language modeling can improve numerous …

In this paper, we propose a novel architecture, the Enhanced Interactive Transformer (EIT),
to address the issue of head degradation in self-attention mechanisms. Our approach …

The compressive strength of concrete is a crucial parameter in structural design, yet its
determination in a laboratory setting is both time-consuming and expensive. The prediction …