The Western Sky Palace, Heavenly Cult, Longevity Qin Family, and other supreme forces harbored some skepticism towards the claims about White Emperor City. According to their analysis of intelligence on Han Li, his personality didn't quite seem like one that would commit such acts.
While Han Li did have the bad habit of plundering beautiful female cultivators to become his Dao companions, he had never directly attacked a force to snatch someone by force; he always acted covertly.
Therefore, they suspected that this might be a frame-up by White Emperor City, meant to cover something up, but they also couldn't uncover the truth.
On the other hand, Empress Yuanchu and the Huangyue Imperial Empress believed it, as Han Li had a very bad image in their hearts.