The fire control solutions; and torp reload times would be painful.
Yes, it typically would take several minutes to reload, and doing so made noise. However, this was mitigated by the fact that a fleet boat generally had fore and aft torpedo rooms (6 forward, 4 aft, was a typical arrangement, IIRC). And at least you
could reload, unlike a PT boat. There would have to be some automation of the analog computer (can't recall the actual name of the US model, a "TDC" or something like that) used for torpedo targeting; a virtual torpedo officer, as it were. Basically, you estimated the targets course and speed from observations, input your boat's course and speed, and finally the bearing to the target as read off the periscope markings. The TDC would output a recommended course and relative bearing at which to shoot. You'd train the periscope out to that bearing and when the target hit your cross-hairs, you fired your torps. Typically, you'd fire a spread of several torps a few seconds apart, in hopes that one would hit the target. At least, that's what I recall from reading about it.