End-to-end transformer-based models in textual-based NLP
A Rahali, MA Akhloufi - AI, 2023 - mdpi.com
Transformer architectures are highly expressive because they use self-attention
mechanisms to encode long-range dependencies in the input sequences. In this paper, we …
mechanisms to encode long-range dependencies in the input sequences. In this paper, we …
Vision-language models in remote sensing: Current progress and future trends
The remarkable achievements of ChatGPT and Generative Pre-trained Transformer 4 (GPT-
4) have sparked a wave of interest and research in the field of large language models …
4) have sparked a wave of interest and research in the field of large language models …
Efficient and effective text encoding for chinese llama and alpaca
Large Language Models (LLMs), such as ChatGPT and GPT-4, have dramatically
transformed natural language processing research and shown promising strides towards …
transformed natural language processing research and shown promising strides towards …
Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation
Pre-trained models have achieved state-of-the-art results in various Natural Language
Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up …
Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up …
Towards artificial general intelligence via a multimodal foundation model
The fundamental goal of artificial intelligence (AI) is to mimic the core cognitive activities of
human. Despite tremendous success in the AI research, most of existing methods have only …
human. Despite tremendous success in the AI research, most of existing methods have only …
Lilt: A simple yet effective language-independent layout transformer for structured document understanding
Structured document understanding has attracted considerable attention and made
significant progress recently, owing to its crucial role in intelligent document processing …
significant progress recently, owing to its crucial role in intelligent document processing …
Chinese clip: Contrastive vision-language pretraining in chinese
The tremendous success of CLIP (Radford et al., 2021) has promoted the research and
application of contrastive learning for vision-language pretraining. In this work, we construct …
application of contrastive learning for vision-language pretraining. In this work, we construct …
Cpt: A pre-trained unbalanced transformer for both chinese language understanding and generation
In this paper, we take the advantage of previous pre-trained models (PTMs) and propose a
novel Chinese pre-trained unbalanced transformer (CPT). Different from previous Chinese …
novel Chinese pre-trained unbalanced transformer (CPT). Different from previous Chinese …
Chinesebert: Chinese pretraining enhanced by glyph and pinyin information
Recent pretraining models in Chinese neglect two important aspects specific to the Chinese
language: glyph and pinyin, which carry significant syntax and semantic information for …
language: glyph and pinyin, which carry significant syntax and semantic information for …
CLUE: A Chinese language understanding evaluation benchmark
The advent of natural language understanding (NLU) benchmarks for English, such as
GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of …
GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of …