Very true. However, with very young children you always have to secure the guns in some way regardless.
Guns should be secured regardless.
There is more of a chance that a neighbor kid will get their hands on it than your own... IF the kids have been taught properly. My kids wouldn't dare touch a gun anywhere (ages 7 and 10) regardless of where they're at if I or their grandfather's were not around. Both of my kids know gun safety better than %95 of adults. I'm not joking, either.
I continuously teach and reinforce what I've taught to my kids the importance of gun safety, AND the importance of respecting firearms. If someone gets out even a BB gun they are to leave immediately.