Leonard knew from the very beginning that he would be faced with rejection since he, himself, thought that this was a very bad idea. Not only for the prince himself but also for the lady's image. However, the most important thing would of course be his master's reputation. The issue in this crazy idea lied in the timing and the person whom the second prince wished to propose to.
Forget about Lady Alice's unsavory reputation among the high society circle and the citizen itself, just the fact that she was Ophelia Brooks' cousin was already enough to get people talking. That was a big question mark on why the prince would choose the other girl afterwards.
But, the biggest possibility would be Ophelia's incident and the moment that thought crossed the people's mind, they would immediately conclude that the prince had left Ophelia at her worst. Although what the lady had done sounded reasonable for an excuse to break up, the idea of romance killed any logic.