One vital point in this is the wind gradient. Whatever the wind is doing on the ground, it is doing more of it at altitude. Because your aircraft has inertia, it does not simply ignore this change in velocity, it responds to it as an acceleration. Climbing into wind, this acceleration is positive and your aircraft gains energy from the airmass as it climbs, and thus climbs faster. Climbing downwind the accelleration is negative, you lose energy and you climb slower. The reverse is true when descending, which is why you'd better carry a little extra on approach, because the wind gradient can leave you short of both airspeed and altitude in a hurry if you don't.
To visualize this, imagine an absolutely sharp wind shear layer of twenty knots. Below it, you're cruising along straight and level. Climb two feet, the wings are in the new airmass and they're flying twenty knots faster than they were a second ago. That extra airspeed immediately turns into extra altitude as your aircraft (still trimmed for S&L twenty knots slower) slows down.
With a sharp enough shear a skilled glider pilot can maintain altitude simply by pulling up and turning into the gradient and then pushing over and turning out of it. Such shears are rare, but condors use the same effect close to the ocean to soar for hours without flapping a wing.