The president's threat, which has sparked bipartisan pushback, could play big with swing voters this November.