Li Yao was utterly oblivious to the fact that he had become a person of interest, with people sent out to find him.
Of course, even if he did know, he wouldn't care. Not to mention that Li Yao wouldn't be afraid of them grouping up against him. Considering this situation, if they dared to gather so many people, if discovered by the Ogres, not even Li Yao would need to lift a finger as the Ogres would send their troops to eradicate them.
At the Warcraft Level, monstrous creatures are no longer confined to their territories. They won't just foolishly wait for you to attack. If they sense a threat, they'll have their ways of dealing with it.
This is the real reason why Li Yao has been so cautious towards Warcrafts. Attempting to charge into an enemy camp is a surefire path to suicide.
However, now is the time of trials, the time when the enemy camp is most vulnerable. If a few days go by, there may not be this good an opportunity again.