The problem is that people can only see the damage caused by the Avengers. Because the far greater catastrophe was averted, then it is fairly simple for people to question whether said catastrophe actually was imminent at all or if it was just an excuse for the Enhanced to flex their muscles. The harm seen always trumps the potential harm averted in the mind of the short-sighted and, regrettably, the vast majority of leaders are short-sighted.
I'm really hoping that we see more of Tony's growing guilt complex, for what it's worth. He's clearly punishing himself by one remove by punishing his fellow Avengers. Worse, he's see-sawing between extremes of freedom and regulation; as each fails him in turn, his snap in the other direction is growing more violent. A final, violent breakdown may be inevitable.