Typically, the secondary armament on a major WWII warship was controlled by special secondary armament directors, which were mounted high on the superstructure, away from the gun mounts. This would normally tend to avoid any smoke problem from those guns.
It is likely that secondary mounts were physically limited from training to bearings and elevations which pointed them too close to ship structure, both to avoid directly hitting structure, and to avoid blast damage.