Ghent University Academic Bibliography

Advanced

Training energy-based models for time-series imputation

Philémon Brakel, Dirk Stroobandt UGent and Benjamin Schrauwen (2013) JOURNAL OF MACHINE LEARNING RESEARCH. 14. p.2771-2797
abstract
Imputing missing values in high dimensional time-series is a difficult problem. This paper presents a strategy for training energy-based graphical models for imputation directly, bypassing difficulties probabilistic approaches would face. The training strategy is inspired by recent work on optimization-based learning (Domke, 2012) and allows complex neural models with convolutional and recurrent structures to be trained for imputation tasks. In this work, we use this training strategy to derive learning rules for three substantially different neural architectures. Inference in these models is done by either truncated gradient descent or variational mean-field iterations. In our experiments, we found that the training methods outperform the Contrastive Divergence learning algorithm. Moreover, the training methods can easily handle missing values in the training data itself during learning. We demonstrate the performance of this learning scheme and the three models we introduce on one artificial and two real-world data sets.
Please use this url to cite or link to this publication:
author
organization
year
type
journalArticle (original)
publication status
published
subject
keyword
missing values, optimization, neural networks, energy-based models, EXPERTS, time-series
journal title
JOURNAL OF MACHINE LEARNING RESEARCH
volume
14
pages
2771 - 2797
Web of Science type
Article
Web of Science id
000327007400009
JCR category
AUTOMATION & CONTROL SYSTEMS
JCR impact factor
2.853 (2013)
JCR rank
8/59 (2013)
JCR quartile
1 (2013)
ISSN
1532-4435
language
English
UGent publication?
yes
classification
A1
copyright statement
I have transferred the copyright for this publication to the publisher
id
4162809
handle
http://hdl.handle.net/1854/LU-4162809
alternative location
http://jmlr.org/papers/volume14/brakel13a/brakel13a.pdf
date created
2013-10-15 13:49:58
date last changed
2016-12-19 15:40:32
@article{4162809,
  abstract     = {Imputing missing values in high dimensional time-series is a difficult problem. This paper presents a strategy for training energy-based graphical models for imputation directly, bypassing difficulties probabilistic approaches would face. The training strategy is inspired by recent work on optimization-based learning (Domke, 2012) and allows complex neural models with convolutional and recurrent structures to be trained for imputation tasks. In this work, we use this training strategy to derive learning rules for three substantially different neural architectures. Inference in these models is done by either truncated gradient descent or variational mean-field iterations. In our experiments, we found that the training methods outperform the Contrastive Divergence learning algorithm. Moreover, the training methods can easily handle missing values in the training data itself during learning. We demonstrate the performance of this learning scheme and the three models we introduce on one artificial and two real-world data sets.},
  author       = {Brakel, Phil{\'e}mon and Stroobandt, Dirk and Schrauwen, Benjamin},
  issn         = {1532-4435},
  journal      = {JOURNAL OF MACHINE LEARNING RESEARCH},
  keyword      = {missing values,optimization,neural networks,energy-based models,EXPERTS,time-series},
  language     = {eng},
  pages        = {2771--2797},
  title        = {Training energy-based models for time-series imputation},
  url          = {http://jmlr.org/papers/volume14/brakel13a/brakel13a.pdf},
  volume       = {14},
  year         = {2013},
}

Chicago
Brakel, Philémon, Dirk Stroobandt, and Benjamin Schrauwen. 2013. “Training Energy-based Models for Time-series Imputation.” Journal of Machine Learning Research 14: 2771–2797.
APA
Brakel, P., Stroobandt, D., & Schrauwen, B. (2013). Training energy-based models for time-series imputation. JOURNAL OF MACHINE LEARNING RESEARCH, 14, 2771–2797.
Vancouver
1.
Brakel P, Stroobandt D, Schrauwen B. Training energy-based models for time-series imputation. JOURNAL OF MACHINE LEARNING RESEARCH. 2013;14:2771–97.
MLA
Brakel, Philémon, Dirk Stroobandt, and Benjamin Schrauwen. “Training Energy-based Models for Time-series Imputation.” JOURNAL OF MACHINE LEARNING RESEARCH 14 (2013): 2771–2797. Print.