Cross-modal audio-visual perception has been a long-lasting topic in psychology and
neurology, and various studies have discovered strong correlations in human perception of …

Visual and audio modalities are two symbiotic modalities underlying videos, which contain
both common and complementary information. If they can be mined and fused sufficiently …

Abstract Multi-instrument Automatic Music Transcription (AMT), or the decoding of a musical
recording into semantic musical content, is one of the holy grails of Music Information …

Can we perform an end-to-end music source separation with a variable number of sources
using a deep learning model? This paper presents an extension of the Wave-U-Net [1] …

Western classical music comprises a rich repertoire composed for different ensembles.
Often, these ensembles consist of instruments from one or two of the families woodwinds …

6G 时代下, 为了兼顾多媒体用户音频, 视频, 触觉的沉浸式体验需求与低时延, 高可靠,
大容量的通信质量, 提出一种跨模态信号重建架构和由视频信号重建触觉信号的深度学习模型 …

Both audio and visual aspects of a musical performance, especially their association, are
important for expressing players' ideas and for engaging the audience. In this paper, we …

In this paper, we propose a multimodal transformer using parallel concatenated architecture.
Instead of using patches, we use column stripes for images in R, G, B channels as the …

With the proliferation of video content of musical performances, audio-visual analysis
becomes an emerging topic in music information retrieval. Associating the audio and visual …

In this work we propose novel joint and sequential multimodal approaches for the task of
single channel audio source separation in videos. This is done within the popular non …