Although the people of the Celestial King Planet could come to a low-level planet like Earth, their actions here were restricted. Just like Gerald, before he came, he had to sign a guarantee that he would not kill anyone on Earth, disturb the order that Earth should have, or affect Earth in any way.
The higher the level of civilization, the more kind they had to be, and the more they would know about dealing with things from low-level civilizations in the most appropriate way.
For example, the richer the family, the more they took good care of low-level animals, such as cats and dogs. This was because these low-level animals could bring them mental joy.
Similarly, it was the same for the Earth and Celestial King Planet.
Naturally, they could destroy Earth and come to Earth to do whatever they wanted. However, if they really did that, Earth would no longer be Earth and they would have one less place to have a vacation.