With the growing interest in sports events, the ability to capture highlights has become increasingly important. Traditionally, the process of editing these highlights required significant time and manpower. To address this challenge, this paper introduces an innovative multi-modal deep learning method for highlight detection (MMDL). The proposed MMDL integrates information from multiple modalities, including subtitles, static skeletal features, and video content, to gain a deep understanding of specific behaviors and identify sub-videos containing those highlights. Additionally, the proposed MMDL employed Siamese networks to accurately capture different aspects of behavior by comparing the similarity between input and training videos across different modalities. Experiments conducted on two datasets, MLB-YouTube and ELTA, demonstrate that the proposed MMDL significantly outperforms existing models, achieving at least a 5% improvement in F1-Score compared to the baseline models, such as I3D and NPL.
關聯:
International Journal of Multimedia Information Retrieval 14(18)
