Pa(sic)-HuBERT: SELF-SUPERVISED MUSIC SOURCE SEPARATION VIA PRIMITIVE AUDITORY CLUSTERING AND HIDDEN-UNIT BERT

被引:1
作者
Chen, Ke [1 ,2 ]
Wichern, Gordon [1 ]
Germain, Francois G. [1 ]
Le Roux, Jonathan [1 ]
机构
[1] MERL, Cambridge, MA 02139 USA
[2] UCSD, La Jolla, CA 92093 USA
来源
2023 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING WORKSHOPS, ICASSPW | 2023年
关键词
Music source separation; primitive auditory principles; self-supervised Learning; BERT; AUDIO REPRESENTATIONS; EXTRACTION; MODEL;
D O I
10.1109/ICASSPW59220.2023.10193575
中图分类号
O42 [声学];
学科分类号
070206 ; 082403 ;
摘要
In spite of the progress in music source separation research, the small amount of publicly-available clean source data remains a constant limiting factor for performance. Thus, recent advances in self-supervised learning present a largely-unexplored opportunity for improving separation models by leveraging unlabelled music data. In this paper, we propose a self-supervised learning framework for music source separation inspired by the HuBERT speech representation model. We first investigate the potential impact of the original HuBERT model by inserting an adapted version of it into the well-known Demucs V2 time-domain separation architecture. We then propose Pa(sic)-HuBERT, a time-frequency-domain self-supervised model, that we later use in combination with a ResU-Net decoder for source separation. Pa(sic)-HuBERT uses primitive auditory features of music as unsupervised clustering labels to initialize the self-supervised pretraining process using the Free Music Archive (FMA) dataset. The resulting framework achieves better source-to-distortion ratio (SDR) performance on the MusDB18 test set than the original Demucs V2 and Res-U-Net models. We further demonstrate that it can boost performance with small amounts of supervised data. Ultimately, our proposed framework is an effective solution to the challenge of limited clean source data for music source separation.
引用
收藏
页数:5
相关论文
共 45 条
[41]  
Vaswani A, 2017, ADV NEUR IN, V30
[42]   WAV2CLIP: LEARNING ROBUST AUDIO REPRESENTATIONS FROM CLIP [J].
Wu, Ho-Hsiang ;
Seetharaman, Prem ;
Kumar, Kundan ;
Bello, Juan Pablo .
2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, :4563-4567
[43]   MULTI-TASK SELF-SUPERVISED PRE-TRAINING FOR MUSIC CLASSIFICATION [J].
Wu, Ho-Hsiang ;
Kao, Chieh-Chi ;
Tang, Qingming ;
Sun, Ming ;
McFee, Brian ;
Bello, Juan Pablo ;
Wang, Chao .
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021), 2021, :556-560
[44]  
Zhao XY, 2022, ASIAPAC SIGN INFO PR, P330, DOI 10.23919/APSIPAASC55919.2022.9980218
[45]   MusicBERT: A Self-supervised Learning of Music Representation [J].
Zhu, Hongyuan ;
Niu, Ye ;
Fu, Di ;
Wang, Hao .
PROCEEDINGS OF THE 29TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2021, 2021, :3955-3963