A year passed since Sam started this vacation in which he fought different people. On the journey, he saw a lot of people apart from the one he fought.
He saw the life of normal people without any form of great responsibilities burdening them living their lives.
He saw people who have too much burden on them to carry it alone smilingly carry it.
In both his lives, Sam is not a 'normal' person. He has very limited emotions and he was exposed to very few of them. In both of his lives, he was in a race to gain more power, more recognition, more safety. He never stopped to think about what it is doing to his mental state.
He knew that in his current condition, he wouldn't lose that easily, but his mind doesn't change and it hinders his growth when he needed to grow the most? This is the question that came to his mind when he found out about his own inferiority complex when he lost to Gran and tamas.