After the Ninth Princess was brought into the Young Master Manor, Yan Jiuchao launched a purge of the Empress's faction.
Back then, he did not get her out of the Cold Palace out of his feelings for her. He did not even think highly of her. She was just a chess piece to keep Worthy Consort Xu in check. Worthy Consort Xu had already lost her power, and this chess piece was no longer of any value. It was not a pity to pull it out. However, Yan Jiuchao was not someone who would kick someone to the curb when they've outlived their usefulness. As long as the Empress did not go overboard, Yan Jiuchao would not interfere whether she was glorious or down and out in the palace.
However, she was courting death and even treated Yan Jiuchao as a bad judge of character. That was unbearable.