Abstract
Choosing the best hyperparameters for neural networks is a big challenge. This paper proposes a method that automatically initializes and adjusts hyperparameters during the training process of stacked autoencoders. A population of autoencoders is trained with gradient-descent-based weight updates, while hyperparameters are mutated and weights are inherited in a Lamarckian kind of way. The training is conducted layer-wise, while each new layer initiates a new neuroevolutionary optimization process. In the fitness function of the evolutionary approach a dimensionality reduction quality measure is employed. Experiments show the contribution of the most significant hyperparameters, while analyzing their lineage during the training process. The results confirm that the proposed method outperforms a baseline approach on MNIST, FashionMNIST, and the Year Prediction Million Song Database.
Original language | English |
---|---|
Title of host publication | 2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Proceedings |
Number of pages | 8 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Publication date | 2019 |
Pages | 823-830 |
Article number | 8790182 |
ISBN (Electronic) | 9781728121536 |
DOIs | |
Publication status | Published - 2019 |
Event | 2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Wellington, New Zealand Duration: 10 Jun 2019 → 13 Jun 2019 |
Conference
Conference | 2019 IEEE Congress on Evolutionary Computation, CEC 2019 |
---|---|
Country/Territory | New Zealand |
City | Wellington |
Period | 10/06/2019 → 13/06/2019 |
Sponsor | et al., Facebook, IEEE, IEEE CIS, Tourism New Zealand, Victoria University of Wellington |
Keywords
- autoencoder
- hyperparameter tuning
- neuroevolution