Before going for Dark Castle, Angor first returned to Centenary Palace and put Fiona inside his space bracelet.
He didn't fully trust the Grand Priest. Leaving Fiona behind meant leaving an unwanted surprise.
With Fiona secured, Angor came to the altar again. But instead of heading to the portal, he first took Toby onto the holy mountain.
Soon, Toby was "dream walking" by fluttering along the mountain slope while sleeping.
Angor followed behind until they stopped in the middle of the hillside. From here, going up or down would cause Toby to struggle again. Angor tried walking at the same altitude until he saw a small path behind the mountain which was connected to the temple.
The other end of the path ended at a cliff.
Angor could sense that the cliff was actually created by an illusion, behind which lied a deep cave. He didn't need to approach the cave to sense a dangerous aura coming from it.