Shouldn't the Crown Prince and the Crown Princess Consort be living in one of the palaces within the Imperial Palace? Why would they talk about going back to their residence? Could it be that they didn't live in the Imperial Palace?
What surprised Long Yuyao even more was that after the man returned, the family had dinner together, played with the siblings for a while, and each of the three Emperors retired to their own palaces to sleep, while the man stayed overnight in a side hall.
The woman didn't refuse, and even had their sibling's little beds moved to the side of her own, saying that she could see them at any time.
Long Yuyao and Yu Siyue had no children, but he had heard that other Princes' women, including the most favored ones who had given birth, would not share a room with that Prince. Even if they desired to stay, the women would decline, let alone this man taking care of this woman and their siblings at night.