Ya, I've always had issues with sweep picking. Left hand hits the right frets, right hand not so great.
You have the technique, but not the coordination. This does take time, perhaps 2-6 weeks. Do the arpaggio at 60 bpm hitting quarter notes. Speed is a non-factor in this excercise.
Then after you hit your quarter notes, move up to 1/8 notes (hitting a note between pings). Then, 1/16th notes. Once you have the 1/16 notes down, then move to 8 bpm up in increments until you confortably get to 120 bpm. This could take 4-8 weeks so be patient.
Concentrate on accuracy and coordination. Then you will be able to do this song start-end with no issues.