Evolution of Stacked Autoencoders

Tim Silhan, Stefan Oehmcke*, Oliver Kramer

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

3 Citations (Scopus)

Abstract

Choosing the best hyperparameters for neural networks is a big challenge. This paper proposes a method that automatically initializes and adjusts hyperparameters during the training process of stacked autoencoders. A population of autoencoders is trained with gradient-descent-based weight updates, while hyperparameters are mutated and weights are inherited in a Lamarckian kind of way. The training is conducted layer-wise, while each new layer initiates a new neuroevolutionary optimization process. In the fitness function of the evolutionary approach a dimensionality reduction quality measure is employed. Experiments show the contribution of the most significant hyperparameters, while analyzing their lineage during the training process. The results confirm that the proposed method outperforms a baseline approach on MNIST, FashionMNIST, and the Year Prediction Million Song Database.

Original languageEnglish
Title of host publication2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Proceedings
Number of pages8
PublisherInstitute of Electrical and Electronics Engineers Inc.
Publication date2019
Pages823-830
Article number8790182
ISBN (Electronic)9781728121536
DOIs
Publication statusPublished - 2019
Event2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Wellington, New Zealand
Duration: 10 Jun 201913 Jun 2019

Conference

Conference2019 IEEE Congress on Evolutionary Computation, CEC 2019
Country/TerritoryNew Zealand
CityWellington
Period10/06/201913/06/2019
Sponsoret al., Facebook, IEEE, IEEE CIS, Tourism New Zealand, Victoria University of Wellington

Keywords

  • autoencoder
  • hyperparameter tuning
  • neuroevolution

Cite this