I inadvertently up-voted the wrong answer (it happened because of scrolling). The answer now had a total of 1 vote (mine). To cancel my upvote, I clicked down-vote. Unexpectedly the question now had a score of minus one. I don't want this either because the answer isn't so bad that I want to penalise it.
This is very peculiar maths.
1 - 1 = -1
-1 + 1 = 1
Why can't I simply do what I want to do and cancel my vote?