That night after J and Jacob left Damien's boat, Damien left the village as well with his boat. He sailed that night towards Lotus City.
While everyone in the village thought that he was just out in the sea or was fishing if they could not find his boat moored in the harbor, he was actually in Lotus City. Aside from the fact that Damien was more comfortable navigating the boat than driving a car, that kind of transportation was faster and less suspicious.
While the drive back to Lotus City might take him two to three hours, sailing the sea to Lotus City would only take him thirty minutes.
People will not suspect or question where he was going or where he went because they would only think that he was just fishing or out in the sea asleep if he left with his boat. Unlike when he takes his car with him, people will question him because it was so unlike him.