Experimental data show that biological synapses behave quite
differently from the symbolic synapses in all common artificial
neural network models. Biological synapses are dynamic, i.e., their
``weight'' changes on a short time scale by several hundred percent
in dependence of the past input to the synapse. In this article we
address the question how this inherent synaptic dynamics -- which
should not be confused with long term ``learning'' -- affects the
computational power of a neural network. In particular we analyze
computations on temporal and spatio-temporal patterns, and we give a
complete mathematical characterization of all filters that can be
approximated by feedforward neural networks with dynamic synapses.
It turns out that even with just a single hidden layer such networks
can approximate a very rich class of nonlinear filters: all filters
that can be characterized by Volterra series. This result is robust
with regard to various changes in the model for synaptic dynamics.
Our characterization result provides for all nonlinear filters that
are approximable by Volterra series a new complexity hierarchy which
is related to the cost of implementing such filters in neural
systems.