Hypothermia was probably the biggest killer, Sharks a close 2nd. Even in 80* water it only takes a short time for the body's core temp to drop low enough to cause seizures and unconciousness.
If you have ever seen a group of sharks in a feeding frenzy, you can just imagine how many men met their fate that way.