Advanced search
1 file | 2.68 MB Add to list

Investigating the biological relevance in trained embedding representations of protein sequences

Jasper Zuallaert (UGent) , Xiaoyong Pan (UGent) , Yvan Saeys (UGent) , Xi Wang and Wesley De Neve (UGent)
Author
Organization
Abstract
As genome sequencing is becoming faster and cheaper, an abundance of DNA and protein sequence data is available. However, experimental annotation of structural or functional information develops at a much slower pace. Therefore, machine learning techniques have been widely adopted to make accurate predictions on unseen sequence data. In recent years, deep learning has been gaining popularity, as it allows for effective end-to-end learning. One consideration for its application on sequence data is the choice for a suitable and effective sequence representation strategy. In this paper, we investigate the significance of three common encoding schemes on the multi-label prediction problem of Gene Ontology (GO) term annotation, namely a one-hot encoding, an ad-hoc trainable embedding, and pre-trained protein vectors, using different hyper-parameters. We found that traditional unigram one-hot encodings achieved very good results, only slightly outperformed by unigram ad-hoc trainable embeddings and bigram pre-trained embeddings (by at most 3%for the F maxscore), suggesting the exploration of different encoding strategies to be potentially beneficial. Most interestingly, when analyzing and visualizing the trained embeddings, we found that biologically relevant (dis)similarities between amino acid n-grams were implicitly learned, which were consistent with their physiochemical properties.

Downloads

  • DS253.pdf
    • full text (Published version)
    • |
    • open access
    • |
    • PDF
    • |
    • 2.68 MB

Citation

Please use this url to cite or link to this publication:

MLA
Zuallaert, Jasper, et al. “Investigating the Biological Relevance in Trained Embedding Representations of Protein Sequences.” Workshop on Computational Biology at ICML2019, Proceedings, 2019.
APA
Zuallaert, J., Pan, X., Saeys, Y., Wang, X., & De Neve, W. (2019). Investigating the biological relevance in trained embedding representations of protein sequences. In Workshop on Computational Biology at ICML2019, Proceedings. Long Beach, CA.
Chicago author-date
Zuallaert, Jasper, Xiaoyong Pan, Yvan Saeys, Xi Wang, and Wesley De Neve. 2019. “Investigating the Biological Relevance in Trained Embedding Representations of Protein Sequences.” In Workshop on Computational Biology at ICML2019, Proceedings.
Chicago author-date (all authors)
Zuallaert, Jasper, Xiaoyong Pan, Yvan Saeys, Xi Wang, and Wesley De Neve. 2019. “Investigating the Biological Relevance in Trained Embedding Representations of Protein Sequences.” In Workshop on Computational Biology at ICML2019, Proceedings.
Vancouver
1.
Zuallaert J, Pan X, Saeys Y, Wang X, De Neve W. Investigating the biological relevance in trained embedding representations of protein sequences. In: Workshop on Computational Biology at ICML2019, Proceedings. 2019.
IEEE
[1]
J. Zuallaert, X. Pan, Y. Saeys, X. Wang, and W. De Neve, “Investigating the biological relevance in trained embedding representations of protein sequences,” in Workshop on Computational Biology at ICML2019, Proceedings, Long Beach, CA, 2019.
@inproceedings{8628909,
  abstract     = {As genome sequencing is becoming faster and cheaper, an abundance of DNA and protein sequence data is available. However, experimental annotation of structural or functional information develops at a much slower pace. Therefore, machine learning techniques have been widely adopted to make accurate predictions on unseen sequence data. In recent years, deep learning has been gaining popularity, as it allows for effective end-to-end learning. One consideration for its application on sequence data is the choice for a suitable and effective sequence representation strategy. In this paper, we investigate the significance of three common encoding schemes on the multi-label prediction problem of Gene Ontology (GO) term annotation, namely a one-hot encoding, an ad-hoc trainable embedding, and pre-trained protein vectors, using different hyper-parameters. We found that traditional unigram one-hot encodings achieved very good results, only slightly outperformed by unigram ad-hoc trainable embeddings and bigram pre-trained embeddings (by at most 3%for the F maxscore), suggesting the exploration of different encoding strategies to be potentially beneficial. Most interestingly, when analyzing and visualizing the trained embeddings, we found that biologically relevant (dis)similarities between amino acid n-grams were implicitly learned, which were consistent with their physiochemical properties.},
  author       = {Zuallaert, Jasper and Pan, Xiaoyong and Saeys, Yvan and Wang, Xi and De Neve, Wesley},
  booktitle    = {Workshop on Computational Biology at ICML2019, Proceedings},
  language     = {eng},
  location     = {Long Beach, CA},
  pages        = {10},
  title        = {Investigating the biological relevance in trained embedding representations of protein sequences},
  url          = {https://sites.google.com/view/icml-compbio-2019/home},
  year         = {2019},
}