I know exactly what your are talking about.  Every year I take a trip into the Minnesota wilderness.  I only take my glasses and only use them when I have to.  
After spending two weeks on a lake fishing without my glasses, I find that my vision is actually getting better from being forced to focus at far away objects.
Course, as soon as I get back to civilization, in go the contacts and I'm blind again.