C++ may be syntactically similar to C# (C sharp), but they are very different languages. I'd probably suggest avoiding C++ - at least initially.
C# is a decent choice for general programming, as is Java if you want to do stuff on anything other than a Windows machine. If you're interested in programming for the web, take a look at PHP.
There are excellent tools for all of those available at little or no cost - at the very least, you'll want to get yourself a text editor that does keyword highlighting and block indentation - I use Textpad on Windows.
More useful than simple text editor is a full integrated development environment such as Visual C# Express, Eclipse (for Java) or Activestate Komodo (for PHP, Perl and a bunch of other languages). Amongst other things, a good IDE will highlight syntax errors and let you do line-by-line debugging.
As for reference books, there are literally thousands out there of varying quality. Read reviews on Amazon and always check the data book was published - don't buy anything more than a couple of years old. With a few notable exceptions, programming books don't age well.