EUSIPCO'2002 banner

Paper data
A new neural network pruning method based on the singular value decomposition and the weight initialisation

Abid Sabeur, Ecole Supérieure des Sciences et Techniques de Tunis
Fnaiech Farhat, Ecole Supérieure des Sciences et Techniques de Tunis
Najim Mohamed, Enseirb, Bordeaux. France

Page numbers in the proceedings:
Volume I pp 364-367

Nonlinear Signal and Systems / Adaptive Methods

Paper abstract
In this paper, we present an efficient procedure to determine the optimal hidden unit number of a feed-forward multi-layer Neural Network (NN) using the singular value decomposition (SVD) taking into account the function to be approximated by the NN and the initial values of the updating weights. The SVD is used to identify and eliminate redundant hidden nodes. Minimizing redundancy gives smaller networks, producing models that generalize better and thus eliminate the need of using cross-validation to avoid overfitting. Using this procedure we obtain a final model with fewer adjustable parameters and more accurate predictions than a network model with a fixed, a priori determined, size. We show these performances by applying this procedure to several problems such as function approximation and image recognition.

A PDF version is available here