[HTML][HTML] Multibench: Multiscale benchmarks for multimodal representation learning
Learning multimodal representations involves integrating information from multiple
heterogeneous sources of data. It is a challenging yet crucial area with numerous real-world …
heterogeneous sources of data. It is a challenging yet crucial area with numerous real-world …
Quantifying & modeling multimodal interactions: An information decomposition framework
The recent explosion of interest in multimodal applications has resulted in a wide selection
of datasets and methods for representing and integrating information from different …
of datasets and methods for representing and integrating information from different …
Review of bioinspired vision-tactile fusion perception (VTFP): From humans to humanoids
Humanoid robots are designed and expected to resemble humans in structure and
behavior, showing increasing application potentials in various fields. Like their biological …
behavior, showing increasing application potentials in various fields. Like their biological …
An overview of differentiable particle filters for data-adaptive sequential Bayesian inference
By approximating posterior distributions with weighted samples, particle filters (PFs) provide
an efficient mechanism for solving non-linear sequential state estimation problems. While …
an efficient mechanism for solving non-linear sequential state estimation problems. While …
See, hear, and feel: Smart sensory fusion for robotic manipulation
Humans use all of their senses to accomplish different tasks in everyday activities. In
contrast, existing work on robotic manipulation mostly relies on one, or occasionally two …
contrast, existing work on robotic manipulation mostly relies on one, or occasionally two …
Vision-force-fused curriculum learning for robotic contact-rich assembly tasks
Contact-rich robotic manipulation tasks such as assembly are widely studied due to their
close relevance with social and manufacturing industries. Although the task is highly related …
close relevance with social and manufacturing industries. Although the task is highly related …
Learning vision-based pursuit-evasion robot policies
Learning strategic robot behavior—like that required in pursuit-evasion interactions—under
real-world constraints is extremely challenging. It requires exploiting the dynamics of the …
real-world constraints is extremely challenging. It requires exploiting the dynamics of the …
High-modality multimodal transformer: Quantifying modality & interaction heterogeneity for high-modality representation learning
Many real-world problems are inherently multimodal, from spoken language, gestures, and
paralinguistics humans use to communicate, to force, proprioception, and visual sensors on …
paralinguistics humans use to communicate, to force, proprioception, and visual sensors on …
Enhancing state estimation in robots: A data-driven approach with differentiable ensemble kalman filters
This paper introduces a novel state estimation framework for robots using differentiable
ensemble Kalman filters (DEnKF). DEnKF is a reformulation of the traditional ensemble …
ensemble Kalman filters (DEnKF). DEnKF is a reformulation of the traditional ensemble …
Intra-and Inter-Modal Curriculum for Multimodal Learning
Multimodal learning has been widely studied and applied due to its improvement over
previous unimodal tasks and its effectiveness on emerging multimodal challenges …
previous unimodal tasks and its effectiveness on emerging multimodal challenges …