Even though the Ying family dying would not cause much of a loss to Great Qin, it would definitely cause Great Qin to be quite displeased, allowing those factions to achieve their goal. Since they were going to die anyways, they might as well take out their anger on the Ying family.
In order to respond to this, the Ying family mobilized all of the armies at its disposal. After all, the various large families held a lot of power in the real world, and they had created their own personal armies.
In response to this, Zhao Fu decided to properly think about it. Even though he could bring his real body into the Heaven Awaken World at any time, allowing him to not worry about this, not everyone could do this.
The Ying family was indeed not very important to Zhao Fu, as it was just the imperial clan of Great Qin. However, there were some people in the Ying family Zhao Fu cared about, and he did not want to see them all die.