Although Weiss knew that Ayame and her team could protect themselves from the sudden attacks of the spectral custodian, it was more cost-effective to use his Barrier Authority compared to using his Game Over Retry Authority if Ayame and her team did fail.
So other than protecting Ayame and her team from the sneak attack of the spectral custodian, Weiss placed his hands behind his back and walked out of the attack range of the spectral custodian.
He decided to leave the rest to Ayame and her team. If they still failed, then he would be quite speechless since the lineup of Ayame's team was very luxurious.
If they even failed, then it would be a skill issue. It's like having a powerful META game character in a MOBA game yet the player lost due to skill issues.
Ayame and her team had a powerful lineup of authorities and game items so Weiss didn't believe that they could lose against the spectral custodian.