The reality of it is you have to make a choice.
You are already locked into a lifestyle; family, job, hobbies, debts, homeownership, etc.. People have links to you and you to them.
Are you really going to make any major changes in those relationships? You still have obligations, you still have to support people.
The idea that "yeah, I'm just going to chuck all this and go do things I want to do" ala "Easy Rider" (Pete Fonda tosses his wristwatch away) isn't an easy decision.
Ask me how I know.