There are several theories. Some think it might have been a mutiny among the crew, where the mutineers then left the ship. Another theory is that they were exposed to some sort of toxic fumes that affected their mental state and made them leave. There's also the idea that they were lured away by some false signal or mirage, believing there was safety elsewhere.
It could be due to some strange phenomenon at sea. For example, a very unusual weather event or a strange sea creature that they misinterpreted as a threat. Also, there might have been a problem with the ship's structure that they thought was much worse than it actually was, so they decided to abandon ship quickly.
One possible reason could be a sudden fear of the ship sinking. Maybe they saw some water coming in and panicked.