Chapter 669- Presentation of Power
Simon's vague answer left a possibility that he might join their faction. As such, unless he did something foolish by himself first, all those factions shouldn't make things difficult for him first.
Of course, they were only extending their favourability towards him because of Oswell. Had it been any other Demon Earl, they would not even get an invitation from these factions. And even if they did, they might not be in a position to reject or shake off the invitation like Simon.
In the latter's case, he was only accepting the invitations from the various factions because Oswell told him to keep playing them off and maintain a neutral stance till the very last second of the event as it would help him later on.
If not, Simon had no interest in joining any faction. It is true that joining a faction secures and increases the survivability of a demon and their dungeon. However, all those benefits are just superficial.