You don't always lose points. When i get hit by players a lot higher then me, they get nothing, and like wise for me when i attack weaker players.
Let me rephrase..When you attack(You're the attacker) and lose, you always lose 1 point, but more can be added if the person is weaker than you.
That's where logic doesn't come into play.