A simple extension of standard neural network models is introduced that
provides a model for neural computations that involve both firing rates and
firing correlations. Such extension appears to be useful since it has been
shown that firing correlations play a significant computational role in
many biological neural systems. Standard neural network models are only
suitable for describing neural computations in terms of firing rates.

The
resulting extended neural network models are still relatively simple, so that
their computational power can be analyzed theoretically. We prove rigorous
separation results, which show that the use of firing
correlations in addition to firing rates can drastically increase the
computational power of a neural network.

On the side one of our separation
results also throws new light on a question that involves
just standard neural network models: We prove that some high-order
sigmoidal neural nets can compute boolean functions which require for their
computation with first-order sigmoidal units a
substantially larger neural net, without imposing
restrictive conditions on the architecture, parameters, or activation
functions of the first-order sigmoidal neural nets.