The inherent challenge of multimodal fusion is to precisely capture the cross-modal
correlation and flexibly conduct cross-modal interaction. To fully release the value of each …

Machine learning models often perform poorly on subgroups that are underrepresented in
the training data. Yet, little is understood on the variation in mechanisms that cause …

Mixup, a simple data augmentation method that randomly mixes two data points via linear
interpolation, has been extensively applied in various deep learning applications to gain …

Pre-trained multi-modal models, such as CLIP, provide transferable embeddings and show
promising results in diverse applications. However, the analysis of learned multi-modal …

Real-world data usually confronts severe class-imbalance problems, where several majority
classes have a significantly larger presence in the training set than minority classes. One …

Miscalibration in deep learning refers to there is a discrepancy between the predicted
confidence and performance. This problem usually arises due to the overfitting problem …

Recent advancements in large vision-language models (LVLMs) have demonstrated
impressive capability in visual information understanding with human language. Despite …

In prediction tasks, there exist features that are related to the label in the same way across
different settings for that task; these are semantic features or semantics. Features with …

Training large deep learning (DL) models with high performance for natural language
downstream tasks usually requires rich-labeled data. However, in a real-world application of …

We introduce eCLIP, an enhanced version of the CLIP model that integrates expert
annotations in the form of radiologist eye-gaze heatmaps. It tackles key challenges in …