The relativized weak pigeonhole principle states that if at least $2n$ out of $n^2$ pigeons fly into $n$ holes, then some hole must be doubly occupied. We prove that every DNF-refutation of the CNF encoding of this principle requires size $2^{(\log n)^{3/2-\epsilon}}$ for every $\epsilon > 0$ and every sufficiently large $n$. By reducing it to the standard weak pigeonhole principle with $2n$ pigeons and $n$ holes, we also show that this lower bound is essentially tight in that there exist DNF-refutations of size $2^{(\log n)^{O(1)}}$ even in R($\log$). For the lower bound proof we need to discuss the existence of unbalanced low-degree bipartite expanders satisfying a certain robustness condition.