After speaking with Yulia, Vincent was able to understand Zemin Clan's stance on his current position.
Right now, he was thinking of accepting the duel, which no one has probably expected since he was blind. Once he did that and even miraculously win the duel, there would be implications that he wouldn't be able to avoid… One of that would be the falling off of his relationship with the Zemin Clan. That's because he would surely announce the break of the engagement himself after the duel.
So he has to weigh his options.
Though he didn't mind breaking off his relationship with them after they gave him the cold shoulder, he can't remain unattached to a powerful organization as that could probably affect his arrival at the Black Tower.
There was no way he would be allowed to just freely walk off to another country. He would need some kind of background or identity there and the Black Tower is definitely the best option he has right now.