The reason why they believed Ye Cang's words was because he seemed to know a lot about the outside world – more than them, at least – making it likely for him to actually have a relationship with a faction outside.
If what Ye Cang was true, that faction would be incredibly terrifying. Just a small team had such immense strength, and if they sent over a few teams, the Dark Demon world would never know peace again.
Even though they still had some doubts, it was best to take what Ye Cang was saying seriously. If that faction really did exist and they stupidly opposed him, that faction might send out a few teams to destroy them like the Vile Dynasty had been destroyed.
The chains from just then had nearly sealed them, and everyone could tell how terrifying they were. Naturally, none of them wanted to try to undo the seal again.