At first I was quite reluctant to downvoting. When you have < 500 rep and getting there took all your spare change (I hope that translates well), you think twice before giving away a single point just because someone wrote a bad post.
This is the same on every SE site BTW - at one point or another, you stop caring much about your reputation score, and just vote as you feel it.
Example: at 951 points, if I can't find two posts to edit (and get to 955), I'll manage to find one bad answer to downvote (and get to 950), just because I like nice round numbers - don't get me wrong, I don't downvote for the heck of it - I'll really try hard to find a post that deserves my wrath be unleashed.
Downvoting questions is different. I don't hold back. Nobody should. A closed question, however, should only be downvoted if you feel it hasn't been punished hard enough by being closed, taking its current net score into account.
A real reason for downvoting CR answers
Psychological instability related to the last digit of one's rep score aside, I think there are answers that, on CR, should be discouraged.
I think we should avoid upvoting "here's your code, better" answers. Moreover, I think these answers should be outright downvoted - CR is about the fishing rod, not about the fish. The problem is that sometimes such answers do have excellent code that's very very good and that does attract upvotes. But to keep CR in shape and all about reviewing code, we should frown upon them, regardless of how good the code is - perhaps write a comment to ask the author "Nice code, where's the review?".