Even before Zhao Mu Fan marry Lao Fei Yan and Lao Min Na he already has few concubines in his harem, for a woman it should give her a big warning sign that the man is anything but faithful, sadly Lao Min Na judgment is rendered by her blind love so she missed this big sign.
Finally, when he got bored with Lao Min Na he took younger women for his harem, it was known to everyone that Zhao Mu Fan is an amorous man combine with his good looks, charisma as crown prince, and his royal background it was expectant that woman throws themselves om his feet.
There is also another problem about Lao Min Na, after two years she still unable to get pregnant then the rumor about her being infertile is spread to the city like a wildfire, and this just adding Zhao Mu Fan dissatisfaction towards her then soon she lose his favor and her world just got worsen than ever be.