We prove a Carbery-Wright style anti-concentration inequality for the unitary Haar measure, by showing that the probability of a polynomial in the entries of a random unitary falling into an $\varepsilon$ range is at most a polynomial in $\varepsilon$. Using it, we show that the scrambling speed of a random quantum circuit is lower bounded: Namely, every input qubit has an influence that is at least exponentially small in depth, on any output qubit touched by its lightcone.

We give three applications of this new scrambling speed lower bound that apply to random quantum circuits with Haar random gates:
$\bullet$ An optimal $\Omega(\log \varepsilon^{-1})$ depth lower bound for $\varepsilon$-approximate unitary designs;
$\bullet$ A polynomial-time quantum algorithm that computes the depth of a bounded-depth circuit, given oracle access to the circuit;
$\bullet$ A polynomial-time algorithm that learns log-depth circuits up to polynomially small diamond distance, given oracle access to the circuit.
The first depth lower bound works against any architecture. The latter two algorithms apply to architectures defined over any geometric dimension, and can be generalized to a wide class of architectures with good lightcone properties.