I thought there was a mechanic from the system that said that no one could get to the other worlds through the portal in the tower. Like when tang first entered the 11th floor I'm pretty sure it was stated there.