Contrastive deep supervision

L Zhang, X Chen, J Zhang, R Dong, K Ma - European Conference on …, 2022 - Springer
… Recently, deep supervision has been proposed to add auxiliary classifiers … deep neural
networks. By optimizing these auxiliary classifiers with the supervised task loss, the supervision

Supervised contrastive learning

P Khosla, P Teterwak, C Wang… - Advances in neural …, 2020 - proceedings.neurips.cc
… The cross-entropy loss is the most widely used loss function for supervised learning of deep
classification models. A number of works have explored shortcomings of this loss, such as …

Self-supervised learning: Generative or contrastive

X Liu, F Zhang, Z Hou, L Mian, Z Wang… - IEEE transactions on …, 2021 - ieeexplore.ieee.org
… Abstract—Deep supervised learning has achieved great success in the last decade. However,
its defects of heavy dependence on manual labels and vulnerability to attacks have driven …

On the effects of self-supervision and contrastive alignment in deep multi-view clustering

DJ Trosten, S Løkse, R Jenssen… - Proceedings of the …, 2023 - openaccess.thecvf.com
… approaches to deep multi-… supervision-based methods for deep MVC, potentially slowing
the progress of the field. To address this, we present DeepMVC, a unified framework for deep

Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm

Y Li, F Liang, L Zhao, Y Cui, W Ouyang, J Shao… - arXiv preprint arXiv …, 2021 - arxiv.org
… the widespread supervision among the … -text contrastive supervision, we fully exploit data
potential through the use of (1) self-supervision within each modality; (2) multi-view supervision

A survey on contrastive self-supervised learning

A Jaiswal, AR Babu, MZ Zadeh, D Banerjee… - Technologies, 2020 - mdpi.com
… Similar to other deep learning methods, contrastive learning employs a variety of optimization
algorithms for training. The training process involves learning the parameters of encoder …

Provable guarantees for self-supervised deep learning with spectral contrastive loss

JZ HaoChen, C Wei, A Gaidon… - Advances in Neural …, 2021 - proceedings.neurips.cc
… -art by relying on the contrastive learning paradigm, which learns … Our work analyzes
contrastive learning without assuming … provable analysis for contrastive learning where …

Adversarial self-supervised contrastive learning

M Kim, J Tack, SJ Hwang - Advances in neural information …, 2020 - proceedings.neurips.cc
… learning is a research direction that delved into the vulnerability of deep networks in the
intrinsic representation space, which we believe is the root cause of fragility of existing …

Contrastive self-supervised learning: review, progress, challenges and future research directions

P Kumar, P Rawat, S Chauhan - International Journal of Multimedia …, 2022 - Springer
… In the last decade, deep supervised learning has had tremendous success. However, its …
Incorporating contrastive learning (CL) for self-supervised learning (SSL) has turned out as …

Self-supervised learning on graphs: Contrastive, generative, or predictive

L Wu, H Lin, C Tan, Z Gao, SZ Li - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
Deep learning on graphs has recently achieved remarkable success on a variety of tasks,
while such success relies heavily on the massive and carefully labeled data. However, precise …