What the minister said was correct. There was a reason why although there was only a single key wielder, there would still be many tower participants who weren't from the same clan, necessarily, as the wielder.
The Epistemic Tower was literally the best key to transcendents throughout the cosmos. If only an astronomically small percentage of martial warriors became transcendents, the number who did without using the Epistemic Tower was an even more unfathomably small percentage of that already small percentage.
This basically meant that if you wanted to have hopes of becoming a transcendent, something that was already nothing but a dream for the vast majority, doing so without the tower would be even worse than a dream. It would be nothing short of foolish.
Knowing this, how could those clans be willing to give it up without a fight? Even if it meant teaming up to suppress the Ragnors, they would do it.