| Emergence of modularity within one sheet of intrinsically active stochastic neurons |
|---|
| Cornelius Weber and Klaus Obermayer |
|
We investigate how modular structure within a neural net of stochastic
neurons can emerge from intrinsic dynamics and from unsupervised learning
of data. The model consists of an input layer and two hierarchically
organized hidden layers and differs from the Helmholtz machine only in
that hidden neurons are not assigned to a single one of the layers prior
to learning. On the basis of a maximum likelihood framework the task is (i) to infer a hidden code from data using the recognition weights and (ii) to generate given input data from the hidden code using the generative weights. Hidden neurons are fully connected via both weight types allowing to code on different hierarchical levels. The hidden neurons are separated into two groups by their intrinsic parameters. One group, designed to be on the highest hierarchical level, is highly spontaneously active and is thus responsible for the initiation of the hidden code. The other group, designed to be on the lower level, is highly responsive to stimulation by input only and thus likely transmits information between the input and the highest level in both directions. Besides demonstrating the emergence of this hierarchical structure, we show using a different data set that a parallel structure emerges which matches the data. Finally, if the net is trained without any input, then a weak hierarchical structure emerges by the differential intrinsic activity. |
| send email to authors |
Last update 29-07-2000, all inquiries to Bernhard Sendhoff