When the gigantic fire arrow was first short at themselves, almost all Flayer soldiers were ready to defend themselves. From the beginning, Rogue had filled the entirety of Strong fort's surroundings with traps. He had done so with the expectation that at least one lord would rebel against him, even with the threat of death.
Those traps were coming to good use now.
The Winter army dares not advance, for they occupied a mound with slightly higher elevation. It gave them perfect wind to let their arrows fly.
Rogue was not stupid, he realized very clearly that the Winterchildren were trying to goad him in to getting out of the castle. It's just that his numbers were more than superior. His army is more than enough to crush the Winterarmy.
Thus, he decided to get out of the castle. Only, he arranged his army right underneath the Strongfort's walls.