Advanced search
2 files | 1.38 MB

Adaptive speaker diarization of broadcast news based on factor analysis

Brecht Desplanques (UGent) , Kris Demuynck (UGent) and Jean-Pierre Martens (UGent)
Author
Organization
Abstract
The introduction of factor analysis techniques in a speaker diarization system enhances its performance by facilitating the use of speaker specific information, by improving the suppression of nuisance factors such as phonetic content, and by facilitating various forms of adaptation. This paper describes a state-of-the-art iVector-based diarization system which employs factor analysis and adaptation on all levels. The diarization modules relevant for this work are: the speaker segmentation which searches for speaker boundaries and the speaker clustering which aims at grouping speech segments of the same speaker. The speaker segmentation relies on speaker factors which are extracted on a frame-by-frame basis using eigenvoices. We incorporate soft voice activity detection in this extraction process as the speaker change detection should be based on speaker information only and we want it to disregard the non-speech frames by applying speech posteriors. Potential speaker boundaries are inserted at positions where rapid changes in speaker factors are witnessed. By employing Mahalanobis distances, the effect of the phonetic content can be further reduced, which results in more accurate speaker boundaries. This iVector-based segmentation significantly outperforms more common segmentation methods based on the Bayesian Information Criterion (BIC) or speech activity marks. The speaker clustering employs two-step Agglomerative Hierarchical Clustering (AHC): after initial BIC clustering, the second cluster stage is realized by either an iVector Probabilistic Linear Discriminant Analysis (PLDA) system or Cosine Distance Scoring (CDS) of extracted speaker factors. The segmentation system is made adaptive on a file-by-file basis by iterating the diarization process using eigenvoice matrices adapted (unsupervised) on the output of the previous iteration. Assuming that for most use cases material similar to the recording in question is readily available, unsupervised domain adaptation of the speaker clustering is possible as well. We obtain this by expanding the eigenvoice matrix used during speaker factor extraction for the CDS clustering stage with a small set of new eigenvoices that, in combination with the initial generic eigenvoices, models the recurring speakers and acoustic conditions more accurately. Experiments on the COST278 multilingual broadcast news database show the generation of significantly more accurate speaker boundaries by using adaptive speaker segmentation which also results in more accurate clustering. The obtained speaker error rate (SER) can be further reduced by another 13% relative to 7.4% via domain adaptation of the CDS clustering. (C) 2017 Elsevier Ltd. All rights reserved.
Keywords
IBCN, Speaker diarization, Speaker segmentation, iVector, Domain adaptation, Factor analysis

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 934.88 KB
  • DS36 i.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 446.19 KB

Citation

Please use this url to cite or link to this publication:

Chicago
Desplanques, Brecht, Kris Demuynck, and Jean-Pierre Martens. 2017. “Adaptive Speaker Diarization of Broadcast News Based on Factor Analysis.” Computer Speech and Language 46: 72–93.
APA
Desplanques, B., Demuynck, K., & Martens, J.-P. (2017). Adaptive speaker diarization of broadcast news based on factor analysis. COMPUTER SPEECH AND LANGUAGE, 46, 72–93.
Vancouver
1.
Desplanques B, Demuynck K, Martens J-P. Adaptive speaker diarization of broadcast news based on factor analysis. COMPUTER SPEECH AND LANGUAGE. 2017;46:72–93.
MLA
Desplanques, Brecht, Kris Demuynck, and Jean-Pierre Martens. “Adaptive Speaker Diarization of Broadcast News Based on Factor Analysis.” COMPUTER SPEECH AND LANGUAGE 46 (2017): 72–93. Print.
@article{8531449,
  abstract     = {The introduction of factor analysis techniques in a speaker diarization system enhances its performance by facilitating the use of speaker specific information, by improving the suppression of nuisance factors such as phonetic content, and by facilitating various forms of adaptation. This paper describes a state-of-the-art iVector-based diarization system which employs factor analysis and adaptation on all levels. The diarization modules relevant for this work are: the speaker segmentation which searches for speaker boundaries and the speaker clustering which aims at grouping speech segments of the same speaker. The speaker segmentation relies on speaker factors which are extracted on a frame-by-frame basis using eigenvoices. We incorporate soft voice activity detection in this extraction process as the speaker change detection should be based on speaker information only and we want it to disregard the non-speech frames by applying speech posteriors. Potential speaker boundaries are inserted at positions where rapid changes in speaker factors are witnessed. By employing Mahalanobis distances, the effect of the phonetic content can be further reduced, which results in more accurate speaker boundaries. This iVector-based segmentation significantly outperforms more common segmentation methods based on the Bayesian Information Criterion (BIC) or speech activity marks. The speaker clustering employs two-step Agglomerative Hierarchical Clustering (AHC): after initial BIC clustering, the second cluster stage is realized by either an iVector Probabilistic Linear Discriminant Analysis (PLDA) system or Cosine Distance Scoring (CDS) of extracted speaker factors. The segmentation system is made adaptive on a file-by-file basis by iterating the diarization process using eigenvoice matrices adapted (unsupervised) on the output of the previous iteration. Assuming that for most use cases material similar to the recording in question is readily available, unsupervised domain adaptation of the speaker clustering is possible as well. We obtain this by expanding the eigenvoice matrix used during speaker factor extraction for the CDS clustering stage with a small set of new eigenvoices that, in combination with the initial generic eigenvoices, models the recurring speakers and acoustic conditions more accurately. Experiments on the COST278 multilingual broadcast news database show the generation of significantly more accurate speaker boundaries by using adaptive speaker segmentation which also results in more accurate clustering. The obtained speaker error rate (SER) can be further reduced by another 13% relative to 7.4% via domain adaptation of the CDS clustering. (C) 2017 Elsevier Ltd. All rights reserved.},
  author       = {Desplanques, Brecht and Demuynck, Kris and Martens, Jean-Pierre},
  issn         = {0885-2308},
  journal      = {COMPUTER SPEECH AND LANGUAGE},
  keywords     = {IBCN,Speaker diarization,Speaker segmentation,iVector,Domain adaptation,Factor analysis},
  language     = {eng},
  pages        = {72--93},
  title        = {Adaptive speaker diarization of broadcast news based on factor analysis},
  url          = {http://dx.doi.org/10.1016/j.csl.2017.05.003},
  volume       = {46},
  year         = {2017},
}

Altmetric
View in Altmetric
Web of Science
Times cited: