Maybe the villagers were being ungrateful for his past sacrifices. Minato was the one who saved the village from the Nine - Tails' attack. If they started to forget his deeds or were misusing the peace he had brought, he would be angry. He might have thought that his efforts were in vain if the villagers didn't uphold the values he fought for.
It could also be that the villagers were involved in some sort of political intrigue or corruption that was endangering the village's future. Minato, as a former Hokage, would be angry if he saw the villagers being part of something that could lead to the downfall of the place he had dedicated his life to protect. He would feel betrayed by those he had once protected.