?url_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Adc&rft.title=Contrastive+separative+coding+for+self-supervised+representation+learning&rft.creator=Wang%2C+J&rft.creator=Lam%2C+MWY&rft.creator=Su%2C+D&rft.creator=Yu%2C+D&rft.description=To+extract+robust+deep+representations+from+long+sequential+modeling+of+speech+data%2C+we+propose+a+self-supervised+learning+approach%2C+namely+Contrastive+Separative+Coding+(CSC).+Our+key+finding+is+to+learn+such+representations+by+separating+the+target+signal+from+contrastive+interfering+signals.+First%2C+a+multi-task+separative+encoder+is+built+to+extract+shared+separable+and+discriminative+embedding%3B+secondly%2C+we+propose+a+powerful+cross-attention+mechanism+performed+over+speaker+representations+across+various+interfering+conditions%2C+allowing+the+model+to+focus+on+and+globally+aggregate+the+most+critical+information+to+answer+the+%22query%22(current+bottom-up+embedding)+while+paying+less+attention+to+interfering%2C+noisy%2C+or+irrelevant+parts%3B+lastly%2C+we+form+a+new+probabilistic+contrastive+loss+which+estimates+and+maximizes+the+mutual+information+between+the+representations+and+the+global+speaker+vector.+While+most+prior+unsupervised+methods+have+focused+on+predicting+the+future%2C+neighboring%2C+or+missing+samples%2C+we+take+a+different+perspective+of+predicting+the+interfered+samples.+Moreover%2C+our+contrastive+separative+loss+is+free+from+negative+sampling.+The+experiment+demonstrates+that+our+approach+can+learn+useful+representations+achieving+a+strong+speaker+verification+performance+in+adverse+conditions.&rft.subject=Speaker+verification%2C+speech+separation%2C+self+attention%2C+contrastive+loss&rft.publisher=IEEE&rft.date=2021&rft.type=Proceedings+paper&rft.language=eng&rft.source=+++++In%3A++Proceedings+of+the+ICASSP+-+2021+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+(ICASSP).++(pp.+pp.+3865-3869).++IEEE+(2021)+++++&rft.format=text&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10154106%2F1%2F2103.00816.pdf&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10154106%2F&rft.rights=open