We study depth-$5$ algebraic circuits over small finite fields with restricted fan-in of the top product gates. We show that there exists an explicit degree-$d$ polynomial $P(\mathbf{x})$ such that any $\Sigma \Pi^{[\mathrm{poly(d)}]} \Sigma \Pi \Sigma$ circuit, computing $P(\mathbf{x})$, over a small finite field, requires size $2^{\Omega(\sqrt{d})}$. Our work builds upon and strengthens the work of [Kumar-Saptharishi'19], who showed $2^{\Omega(\sqrt{d})}$ lower bounds against $\Sigma \Pi^{[\mathcal{O}(\sqrt{d})]} \Sigma \Pi \Sigma$ circuits over small finite fields. It is known that proving $2^{\omega(d^{1/3} \log n)}$ lower bound for $\Sigma \Pi^{[a]} \Sigma \Pi$ circuits with $a = 2^{\mathcal{O}(d^{1/3} \log d)}$, over fields of characteristic $0$, implies VP $\neq$ VNP. In pursuit of this, we also prove superpolynomial lower bounds over small finite fields for $\Sigma \Pi^{[a]} \Sigma \Pi$ circuits where $a = 2^{\mathcal{O}(d^{\lambda} \log d)}$, for any constant $\lambda < 1/3$.

We use evaluations of the shifted partial derivatives to prove our lower bounds. We follow the same outline as [Kumar-Saptharishi'19], but with a more delicate analysis of the complexity measure. We use a family of the Nisan-Wigderson polynomials as a hard polynomial. We show that over small finite fields, setting the parameters of our measure and the hard polynomial with care, the method of shifted partial derivatives can yield lower bounds well beyond the homogeneity restriction on depth-$4$ circuits.

We also show an exponential gap between depth-$3$ and homogeneous depth-$4$ circuits over small finite fields. Previously, only a superpolynomial gap was known using [Chillara-Mukhopadhyay'17] and depth reduction of polynomials in VP until homogeneous depth-$4$. We use the complexity measure of [Grigoriev-Karpinski'98], and we use the Product of the Inner Product polynomial to show the separation.