A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends

J Gui, T Chen, J Zhang, Q Cao, Z Sun… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Deep supervised learning algorithms typically require a large volume of labeled data to
achieve satisfactory performance. However, the process of collecting and labeling such data …

A survey on contrastive self-supervised learning

A Jaiswal, AR Babu, MZ Zadeh, D Banerjee… - Technologies, 2020 - mdpi.com
Self-supervised learning has gained popularity because of its ability to avoid the cost of
annotating large-scale datasets. It is capable of adopting self-defined pseudolabels as …

Improving graph collaborative filtering with neighborhood-enriched contrastive learning

Z Lin, C Tian, Y Hou, WX Zhao - … of the ACM web conference 2022, 2022 - dl.acm.org
Recently, graph collaborative filtering methods have been proposed as an effective
recommendation approach, which can capture users' preference over items by modeling the …

Text and code embeddings by contrastive pre-training

A Neelakantan, T Xu, R Puri, A Radford, JM Han… - arXiv preprint arXiv …, 2022 - arxiv.org
Text embeddings are useful features in many applications such as semantic search and
computing text similarity. Previous work typically trains models customized for different use …

Simcse: Simple contrastive learning of sentence embeddings

T Gao, X Yao, D Chen - arXiv preprint arXiv:2104.08821, 2021 - arxiv.org
This paper presents SimCSE, a simple contrastive learning framework that greatly advances
state-of-the-art sentence embeddings. We first describe an unsupervised approach, which …

Consert: A contrastive framework for self-supervised sentence representation transfer

Y Yan, R Li, S Wang, F Zhang, W Wu, W Xu - arXiv preprint arXiv …, 2021 - arxiv.org
Learning high-quality sentence representations benefits a wide range of natural language
processing tasks. Though BERT-based pre-trained language models achieve high …

DiffCSE: Difference-based contrastive learning for sentence embeddings

YS Chuang, R Dangovski, H Luo, Y Zhang… - arXiv preprint arXiv …, 2022 - arxiv.org
We propose DiffCSE, an unsupervised contrastive learning framework for learning sentence
embeddings. DiffCSE learns sentence embeddings that are sensitive to the difference …

Unsupervised corpus aware language model pre-training for dense passage retrieval

L Gao, J Callan - arXiv preprint arXiv:2108.05540, 2021 - arxiv.org
Recent research demonstrates the effectiveness of using fine-tuned language models~(LM)
for dense retrieval. However, dense retrievers are hard to train, typically requiring heavily …

Clear: Contrastive learning for sentence representation

Z Wu, S Wang, J Gu, M Khabsa, F Sun, H Ma - arXiv preprint arXiv …, 2020 - arxiv.org
Pre-trained language models have proven their unique powers in capturing implicit
language features. However, most pre-training approaches focus on the word-level training …

Towards unsupervised deep graph structure learning

Y Liu, Y Zheng, D Zhang, H Chen, H Peng… - Proceedings of the ACM …, 2022 - dl.acm.org
In recent years, graph neural networks (GNNs) have emerged as a successful tool in a
variety of graph-related applications. However, the performance of GNNs can be …