Theres really nothing to worry about port scans, it happens to all of us more or less.
Mostly the so called script kiddies are doing massive port scans on random IP's.
Pretty much only the systems in dangers are ones already infected through other ways or haven't been properly kept up to date.