Right and wrong is black and white, there is no grey area. There is no line to be crossed because it isn't a line. It is a wall, you cannot be on both sides at the same time.
Brady and the Patriots, "Compromised the integrity of the game". That's all you need. The penalty is based upon how much you wronged the integrity. Whether it actually affected the outcome of the game is not part and parcel to the discussion. "I know we cheated, but we would have won anyway" does not work.
The messages that Brady received from the ball handler at the very least alerted him to the fact that the air pressure in the ball would be manipulated. The right thing to do would have been to report this to the team, at the very least, and there is no record, or at least no discussion of him alerting anyone about this manipulation. Since he knew about this he is implicit.
The ball handler. He had choices too. He chose to reap a personal reward, in the form of gifts and other compensations and putting himself in Bradys pocket as "his Boy". I understand that this is just an employee, not a millionaire superstar, but how much he got paid has little to do with anything. His position of power does. He would have "moved up the ladder" and his actions would surely put him in a position where Brady and the involved Patriots would have at least, "owed him something". The ball handler did not do the right thing. Brady did not do the right thing. The team did not do the right thing because at the very least they failed to police their employees.
This hurt the game. This disrespected the tens of thousands of people that have played the game and will in the future.
There is no grey area.
The penalty was light in my opinion.