Investigating deep neural network internal clustering and generalization properties

The topic of generalization in deep neural networks has long been one that garners a lot of interest from research groups all around the world [1, 2, 3, 4]. Despite all this interest, however, current knowledge about this topic remains limited. In this thesis, we present a novel way of looking at generalization, through the lens of how internal representations at the neuron level relate to the generalization performance of a deep neural network. We present three main contributions on this subject. (i) First, we extend some previous results in the literature about the statistical distribution of weights [5], and introduce a novel concept, the EW product, whose distribution properties have an even stronger relation to generalization performance. (ii) Second, we build on results obtained in [6] that show that neurons act as binary classifiers inside the network, and show that the quality of this binary partition is related to generalization performance as well. (iii) Finally, we show that neurons not only act as binary classifiers, but also as pattern discriminators, and that the quality of this discrimination has strong ties to generalization performance.