While Denise and Tom spent the night rekindling their love for one another, Fred spent the night in his hotel room, nursing his wounds and causing his ill luck. His heart was filled with so much anger and regret.
He regretted taking advantage of Denise, all those years past. He regretted not talking to her after he left Tom's home. He regretted granting Tom an audience in such a public place; knowing that Tom would be upset. He regretted allowing his younger brother to punch him like a kid, without stopping him.
Fred also hated the fact that he allowed Tom to push him into using the wrong words in describing what went down between him and Denise all those years ago.
Before he left the restaurant, he observed a lot of people videorecording the whole embarrassing scene and this got him worried.