When you turn off your computer, then your ISP's router takes up the responsibility for sending the "connection refused" ICMP packet back to the requester.
Switching to another IP from your ISP, will stop the connection attempts to you, but it just means the ISP's router is handling it, or another computer on the ISP's network, which gets your old IP, handles it.
Thus the paranoia is extended to another user that may be running a firewall, and see hundred's of hits on thier computer.
Kazaa and other programs of that ilk, turn your computer into an Internet server. Just like our WEB server here at HTC. Even if I turn off the computer running this bulletin board, the router will get hundreds of hits from all of you that want access to it.
These end up being ICMP messages sent back to the requester to notify the user that the server is not available.
This happens very quickly, almost instantaneous, but the traffic on the pipe is visible and does degrade the overall connection speed.
Now, if you are not serving files, and only grabbing them from the Internet, the damage to your connection is minimized, but if you serve files, then your connection will take a pounding, even after the server program is terminated.
This is all the basic premise of a DOS attack. It is one way hackers attack routers on the Internet, by sending thousands of requests to non-existant IP/port services, which keeps the router flooded with handling the connection attempts.