%0 Conference Proceedings
%T Comparison of Neural Network Optimizers for Relative Ranking Retention Between Neural Architectures
%+ University of Macedonia [Thessaloniki] (UoM)
%A Kyriakides, George
%A Margaritis, Konstantinos
%Z Part 7: Deep Learning - Convolutional ANN
%< avec comité de lecture
%( IFIP Advances in Information and Communication Technology
%B 15th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI)
%C Hersonissos, Greece
%Y John MacIntyre
%Y Ilias Maglogiannis
%Y Lazaros Iliadis
%Y Elias Pimenidis
%I Springer International Publishing
%3 Artificial Intelligence Applications and Innovations
%V AICT-559
%P 272-281
%8 2019-05-24
%D 2019
%R 10.1007/978-3-030-19823-7_22
%K Deep learning
%K Neural architecture search
%K Ranking
%Z Computer Science [cs]Conference papers
%X Autonomous design and optimization of neural networks is gaining increasingly more attention from the research community. The main barrier is the computational resources required to conduct experimental and production project. Although most researchers focus on new design methodologies, the main computational cost remains the evaluation of candidate architectures. In this paper we investigate the feasibility of using reduced epoch training, by measuring the rank correlation coefficients between sets of optimizers, given a fixed number of training epochs. We discover ranking correlations of more than 0.75 and up to 0.964 between Adam with 50 training epochs, stochastic gradient descent with nesterov momentum with 10 training epochs and Adam with 20 training epochs. Moreover, we show the ability of genetic algorithms to find high-quality solutions of a function, by searching in a perturbed search space, given that certain correlation criteria are met.
%G English
%Z TC 12
%Z WG 12.5
%2 https://inria.hal.science/hal-02331350/document
%2 https://inria.hal.science/hal-02331350/file/483292_1_En_22_Chapter.pdf
%L hal-02331350
%U https://inria.hal.science/hal-02331350
%~ IFIP
%~ IFIP-AICT
%~ IFIP-TC
%~ IFIP-WG
%~ IFIP-TC12
%~ IFIP-AIAI
%~ IFIP-WG12-5
%~ IFIP-AICT-559