For recognition in image data, the large number of features can cause an unnecessary increase in the complexity of the chosen classifier. It is important to select from the available features only those that contribute new information. In many cases, an excess of features not only does not aid classification but in fact actively reduces performance. In this paper we use reduced complexity of both classifier and feature set to improve accuracy and speed of computation for the identification of hand-written digits. We show that performances comparable with existing classifiers can be achieved with a 200 fold smaller network