Brewing trouble
Souta could have ignored the small comment easily. He could have moved on, for his feudal lord would look for him soon. After all, it was an inconsequential comment. How would anyone know whether he was lying or not, especially when he spoke so little? He had long learned how to prevent himself from having any behavioral tells or markers for whether he was lying or not.
Logically speaking, the other person was a scammer and had purposefully let him hear the murmur in order to arouse suspicion and doubt, and cause a disturbance in Souta's mental state. Such a vulnerability would then allow them to try to build a logical premise for how and why he should trust them, before, in a seemingly reasonable and logical sequence of events, going on to scam him.