Actually, when the final hop is us (not a router), it means the problem is on the return trip. The routes to and from your computer are asynchronous, meaning the path to the servers is not the path back to your computer from the servers.
Traceroutes/Ping Plots cannot show both paths. When the error is on the return trip, the last or next to the last hop will show problems in the Ping Plot. Those last two hops cannot have packet loss as they are not routers. The server will never lose a packet (nature of the beast) and a properly configured switch does not lose packets either.
When I see the problem is in the last two hops, I run the trace back to the source IP so we can tell where the problem is.