Very easy to fall in that pothole of thinking when playing a game in a world of anonymity (as in no face-to-face).
I bet you would care in this situation
...
(Image removed from quote.)
Try spying here and see what happens. The opposing player might have a very real reaction to the one peeking at the board. The conscious act of spying is the same, whether anonymous or not.
Your analogy is realistic and would work if in fact you were talking about a simple game in a 1 vs 1, but we all know its more like this.

So in essence, if spying is cheating or unethical/unfair play, than what would noe/hording be? Just sayin.
If its part of the game, its a part you must accept as much as the other.