If you're on the fence or just curious, before you jump in, ask the flight school if they offer discovery flights. With those, they'll take you up for ~45 minutes or so, you get to fly a bit, and it usually costs less than a regular lesson. Plus if you decide to go forward you can count it toward you're dual received time.
Here's some rough numbers $ wise for you to figure. (flight school should have some kind of flyer for you with a similar breakout)
Medical - $200 - ? Varies by location/Medical Examiner (You'll need at least a 3rd Class. Do this before starting)
Training costs:(based on national average of 60 hours to PPL Checkride)
Aircraft rental - Multiply their hourly rental rate by 60 (go with the 180hp. you'll thank me later)
Instructor fees - Multiply their hourly instructor rate by 60
Checkride costs - $400 - $800. Varies greatly by location and examiner
Headsets - Ask if they provide them during training. If so use the loaners until you finish training to save some $ until you know what you really want. Otherwise shop around. Bose, Lightspeed, and David Clark are the top brands.
Books / Ground school - Ground School is a good option if they offer it as a class with other folks. Otherwise you're just 1:1 with the instructor. Ask if they have a specific program (lots of places use the Jeppesen kits). I highly recommend getting the "Rod Machado's Private Pilot Handbook".
Once you're "legal", you'll need to figure in costs of staying current. To carry passengers you have to have 3 takeoff/landings within the preceding 90 days. (If you do them at night, it also keeps you night current).
Every 24 calendar months you also have to do a flight review, which is like a mini checkride.
Your medical will need to be renewed at differing dates depending on your age.
Renters Insurance (not the same as house/apartment) should be a consideration. That covers you if something happens to the aircraft that would cause the school to lose $ while it's being fixed.
Flight schools sometimes offer discounts if you buy "Blocks" of time.
For "$100 Hamburger" trips (yeh I know $100 trips don't exist anymore)... you have to figure if it's cheaper to drive or fly. Cost vs time thing. When I was still doing "fun" trips, I worked out that for me if it was about a 3-4 hour drive that was the beak point. That worked out to about a 1 hour flight each direction (about 3 hours rental time). Any longer and it was cheaper to drive/fly commercial. If you go strictly by time, you have to figure in your flight planning, pre/post flight, loading unloading, pax briefing, etc.
Is flying for "fun" fun? Yes
Is it expensive? Very. Especially if you don't stay current or stop flying completely. Then you just wasted the $.
Is it safe? It's a high risk area, but the risks can be managed. I like how Goodspeed said it in The Rock... "The second you don't respect this, it kills you."
There's a lot of real world pilots on here. We all love to hanger talk. Feel free to ask questions and you'll get a lot of answers.