That conversation he had with the three Grimaldis certainly almost drained him. He knew their reaction to Wulfric wouldn't be all that positive. The only reason that they probably didn't react as violently was because he managed to remind them what their priority should be.
Since their priority right now was meeting his mother, that superseded their worry for him. If not for that, he would probably be forced to stay overnight there, where the three would lecture him about how bad a relationship with Wulfric would be.