Toge Inumaki is important too. His special cursed speech ability makes him stand out. Also, the principal of the school, Masamichi Yaga, is likely to be important as he is in a position of authority and has an influence on Gojo and the other students. There could be other characters like the members of different clans that also play roles in the complex web of the story.
I'm not sure specifically what 'kaori visual novel' is about as there could be many different visual novels with this name. It might be about a character named Kaori and her adventures, relationships, or some sort of fictional story set in a particular world.
It's hard to pinpoint exactly where Gojo is at any given time in the manga. His whereabouts change as the story progresses and different scenarios unfold. It could be in a supernatural realm, a mission site, or interacting with other characters.