Automated layer-wise solution for ensemble deep randomized feed-forward neural network
The randomized feed-forward neural network is a single hidden layer feed-forward neural network that enables efficient learning by optimizing only the output weights. The ensemble deep learning framework significantly improves the performance of randomized neural networks. However, the framework’s capabilities are limited by traditional hyper-parameter selection approaches. Meanwhile, different random network architectures, such as the existence or lack of a direct link and the mapping of direct links, can also strongly affect the results. We present an automated learning pipeline for the ensemble deep randomized feed-forward neural network in this paper, which integrates hyper-parameter selection and randomized network architectural search via Bayesian optimization to ensure robust performance. Experiments on 46 UCI tabular datasets show that our strategy produces state-of-the-art performance on various tabular datasets among a range of randomized networks and feed-forward neural networks. We also conduct ablation studies to investigate the impact of various hyper-parameters and network architectures.
Other Information
Published in: Neurocomputing
License: http://creativecommons.org/licenses/by/4.0/
See article on publisher's website: https://dx.doi.org/10.1016/j.neucom.2022.09.148
Funding
Open Access funding provided by the Qatar National Library
History
Language
- English
Publisher
ElsevierPublication Year
- 2022
License statement
This Item is licensed under the Creative Commons Attribution 4.0 International LicenseInstitution affiliated with
- Qatar University
- College of Engineering - QU
- KINDI Center for Computing Research - QU