“…In-domain pre-training for audio. Existing in-domain (audio-only) self-supervised methods can be broadly categorized by the input signal (e.g., raw waveform [32,33,34], frame-level features [35,36,37] or spectrogram patches [18,38]) and the objective used for self-supervision (e.g., contrastive [39,33,40,41,35] or prediction/reconstruction [18,34,37,36]). For example, wav2vec 2.0 [33] takes raw waveform as inputs and exploits contrastive learning to discriminate contextualized representations in different time segments.…”