He said as long as i have my learners on me, and I stay off the road when cars are on it i should be fine
This is an opportunity for you to do something that should serve you well in life or it could lead you astray. The cool thing is it really depends on you and you get to use a little judgment.
Ask yourself:
"Does that seem right?"
I ask you. Does that seem right? Feel free to use logic, follow your instinct, listen to your intuition or trust in the force if it helps you but I'd really like to hear you make an argument for it being a good idea. As an exercise, make an argument for it being a bad idea. Objecitvely look at the facts and make a decision based on said facts, your experience and what you think is right.
Is driving your go-kart on public roads a good idea or a bad idea? Why? Why not?
To get you started on this decision I'd offer you to look into legalities as to what constitutes an automobile, farm equipment, inquire as to why "street legal" is a phrase in common usage and the eligibility and legality requirements for the aforementioned words and phrases. Good luck.