We study two conjectures in additive combinatorics. The first is the polynomial Freiman-Ruzsa conjecture, which relates to the structure of sets with small doubling. The second is the inverse Gowers conjecture for $U^ $, which relates to functions which locally look like quadratics. In both cases a weak form, with exponential decay of parameters is known, and a strong form with only a polynomial loss of parameters is conjectured. Our main result is that the two conjectures are in fact equivalent.