Perhaps the first deep artificial neural network incorporating some neurophysiological insights was the Neocognitron. Recent brain-inspired models of artificial neural networks include especially the so- called Deep Belief Networks and Convolutional Neural Networks. Both types of networks comprise several layers of functional neurons and both of them proved to be able to beat human performance in various areas of 2D image recognition. These models are, however, expected to yield superior results also for many other tasks ranging from language understanding and translation to multimedia data processing, among others.

While the majority of classical image processing techniques is based on carefully preselected image features, deep neural networks are designed to learn local features autonomously with minimum or no advanced pre-processing. The representations formed in their hidden layers resemble a hierarchy combining simpler features found at lower layers into more complex features detected at higher layers. Deep networks can be moreover trained by means of unlabeled data collected, e.g., from the internet. The found features can then be used as common building blocks for new images if labeled data is scarce.

In this thesis, we propose a general framework for training of BP/networks. It is based on the fast and robust scaled conjugate gradient technique. This classical training algorithm is enhanced with analytical or approximative sensitivity inhibition during training and enforcement of a transparent internal knowledge representation. Redundant hidden and input neurons are pruned based on internal representation and sensitivity analysis.

The performance of the developed framework has been tested on various types of data with promising results. The framework provides a fast training algorithm, robust to tunable parameters. Furthermore, it outperforms the reference techniques in the achieved generalization ability and robustness to noise in the data. It is very likely to identify redundant input features and create a simple and transparent network structure during training. In such a way it simplifies knowledge extraction from the model.