Advanced search
1 file | 419.20 KB Add to list

Training energy-based models for time-series imputation

Philémon Brakel (UGent) , Dirk Stroobandt (UGent) and Benjamin Schrauwen (UGent)
Author
Organization
Abstract
Imputing missing values in high dimensional time-series is a difficult problem. This paper presents a strategy for training energy-based graphical models for imputation directly, bypassing difficulties probabilistic approaches would face. The training strategy is inspired by recent work on optimization-based learning (Domke, 2012) and allows complex neural models with convolutional and recurrent structures to be trained for imputation tasks. In this work, we use this training strategy to derive learning rules for three substantially different neural architectures. Inference in these models is done by either truncated gradient descent or variational mean-field iterations. In our experiments, we found that the training methods outperform the Contrastive Divergence learning algorithm. Moreover, the training methods can easily handle missing values in the training data itself during learning. We demonstrate the performance of this learning scheme and the three models we introduce on one artificial and two real-world data sets.
Keywords
missing values, optimization, neural networks, energy-based models, EXPERTS, time-series

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 419.20 KB

Citation

Please use this url to cite or link to this publication:

MLA
Brakel, Philémon, Dirk Stroobandt, and Benjamin Schrauwen. “Training Energy-based Models for Time-series Imputation.” JOURNAL OF MACHINE LEARNING RESEARCH 14 (2013): 2771–2797. Print.
APA
Brakel, P., Stroobandt, D., & Schrauwen, B. (2013). Training energy-based models for time-series imputation. JOURNAL OF MACHINE LEARNING RESEARCH, 14, 2771–2797.
Chicago author-date
Brakel, Philémon, Dirk Stroobandt, and Benjamin Schrauwen. 2013. “Training Energy-based Models for Time-series Imputation.” Journal of Machine Learning Research 14: 2771–2797.
Chicago author-date (all authors)
Brakel, Philémon, Dirk Stroobandt, and Benjamin Schrauwen. 2013. “Training Energy-based Models for Time-series Imputation.” Journal of Machine Learning Research 14: 2771–2797.
Vancouver
1.
Brakel P, Stroobandt D, Schrauwen B. Training energy-based models for time-series imputation. JOURNAL OF MACHINE LEARNING RESEARCH. 2013;14:2771–97.
IEEE
[1]
P. Brakel, D. Stroobandt, and B. Schrauwen, “Training energy-based models for time-series imputation,” JOURNAL OF MACHINE LEARNING RESEARCH, vol. 14, pp. 2771–2797, 2013.
@article{4162809,
  abstract     = {Imputing missing values in high dimensional time-series is a difficult problem. This paper presents a strategy for training energy-based graphical models for imputation directly, bypassing difficulties probabilistic approaches would face. The training strategy is inspired by recent work on optimization-based learning (Domke, 2012) and allows complex neural models with convolutional and recurrent structures to be trained for imputation tasks. In this work, we use this training strategy to derive learning rules for three substantially different neural architectures. Inference in these models is done by either truncated gradient descent or variational mean-field iterations. In our experiments, we found that the training methods outperform the Contrastive Divergence learning algorithm. Moreover, the training methods can easily handle missing values in the training data itself during learning. We demonstrate the performance of this learning scheme and the three models we introduce on one artificial and two real-world data sets.},
  author       = {Brakel, Philémon and Stroobandt, Dirk and Schrauwen, Benjamin},
  issn         = {1532-4435},
  journal      = {JOURNAL OF MACHINE LEARNING RESEARCH},
  keywords     = {missing values,optimization,neural networks,energy-based models,EXPERTS,time-series},
  language     = {eng},
  pages        = {2771--2797},
  title        = {Training energy-based models for time-series imputation},
  url          = {http://jmlr.org/papers/volume14/brakel13a/brakel13a.pdf},
  volume       = {14},
  year         = {2013},
}

Web of Science
Times cited: