Dsanet: Dynamic segment aggregation network for video-level representation learning

W Wu, Y Zhao, Y Xu, X Tan, D He, Z Zou, J Ye… - Proceedings of the 29th …, 2021 - dl.acm.org
W Wu, Y Zhao, Y Xu, X Tan, D He, Z Zou, J Ye, Y Li, M Yao, Z Dong, Y Shi
Proceedings of the 29th ACM International Conference on Multimedia, 2021dl.acm.org
Long-range and short-range temporal modeling are two complementary and crucial aspects
of video recognition. Most of the state-of-the-arts focus on short-range spatio-temporal
modeling and then average multiple snippet-level predictions to yield the final video-level
prediction. Thus, their video-level prediction does not consider spatio-temporal features of
how video evolves along the temporal dimension. In this paper, we introduce a novel
Dynamic Segment Aggregation (DSA) module to capture relationship among snippets. To …
Long-range and short-range temporal modeling are two complementary and crucial aspects of video recognition. Most of the state-of-the-arts focus on short-range spatio-temporal modeling and then average multiple snippet-level predictions to yield the final video-level prediction. Thus, their video-level prediction does not consider spatio-temporal features of how video evolves along the temporal dimension. In this paper, we introduce a novel Dynamic Segment Aggregation (DSA) module to capture relationship among snippets. To be more specific, we attempt to generate a dynamic kernel for a convolutional operation to aggregate long-range temporal information among adjacent snippets adaptively. The DSA module is an efficient plug-and-play module and can be combined with the off-the-shelf clip-based models (i.e., TSM, I3D) to perform powerful long-range modeling with minimal overhead. The final video architecture, coined as DSANet. We conduct extensive experiments on several video recognition benchmarks (i.e., Mini-Kinetics-200, Kinetics-400, Something-Something V1 and ActivityNet) to show its superiority. Our proposed DSA module is shown to benefit various video recognition models significantly. For example, equipped with DSA modules, the top-1 accuracy of I3D ResNet-50 is improved from 74.9% to 78.2% on Kinetics-400. Codes are available at https://github.com/whwu95/DSANet.
ACM Digital Library
以上显示的是最相近的搜索结果。 查看全部搜索结果