# On the huge amount of work needed for a CR answer

I'm a regular at Stack Overflow, and I just recently discovered Code Review, where I answered some questions with the tag. We all know that CR is quite different from SO: by its very nature, both the questions and answers at CR are huge, full of code and explanations. I can't imagine a one-line answer here at CR, although they are quite common (and sometimes heavily upvoted) at SO.

We are programmers trying to help fellow programmers, that's the most important part. However, those green points we (sometimes) receive are an important part of the game, even if most people deny it. And that's why I'm writing this question, which is not exactly a question but a discussion starter instead.

It seems to me that there is a big disproportion between the amount of work we have answering a question at CR and the number of votes we (may) get. For instance, in my case, I wrote 7 answers, totalling around 5000 words. Just for having a better picture of what that means, I decided to pick 7 answers of mine at SO, randomly (I know this lacks scientific validity, it's just for illustrate the issue). Then, comparing both sets of answers, I analysed the number of words and the number of votes in each, creating this bar chart (as a bonus, if you hover over the bars, you'll get a nice red freehand!):

var w=600,h=250,padding=[10,10,100,80],data=[{site:"Stack Overflow",words:1121,votes:22},{site:"Code Review",words:5083,votes:13}],svg=d3.select("body").append("svg").attr("width",w).attr("height",h),xScale=d3.scaleLinear().range([padding[3],w-padding[1]]).domain([0,1.1*d3.max(data,function(t){return t.words/t.votes})]),yScale=d3.scaleBand().range([padding[0],h-padding[2]]).domain(data.map(function(t){return t.site})).padding(.5),colorScale=d3.scaleOrdinal(["#ff7f0e","#1f77b4"]),bars=svg.selectAll(null).data(data).enter().append("rect").style("fill",function(t,e){return colorScale(e)}).attr("x",padding[3]).attr("width",0).attr("y",function(t){return yScale(t.site)}).attr("height",yScale.bandwidth());bars.transition().duration(3e3).attr("width",function(t){return xScale(t.words/t.votes)-padding[3]});var labels=svg.selectAll(null).data(data).enter().append("text").attr("dominant-baseline","central").attr("y",function(t){return yScale(t.site)+yScale.bandwidth()/2}).attr("x",padding[3]+10).style("font-family","sans-serif").style("font-size","12px");labels.transition().duration(3e3).attr("x",function(t){return xScale(t.words/t.votes)+10}).attrTween("text",function(t){var e=this,a=d3.interpolateNumber(0,t.words/t.votes);return function(t){return d3.select(e).text(~~a(t))}});var freehandCoord="m -2.0278636,-4.2995802 c -0.4608758,0.1152189 -0.6108077,0.047127 -0.8690844,0.4345422 -0.026782,0.040174 0.040173,0.1180651 0,0.1448474 -0.1270396,0.084693 -0.2979788,0.076566 -0.4345422,0.1448474 -0.061073,0.030536 -0.083774,0.1143109 -0.1448474,0.1448474 -0.043185,0.021593 -0.1107065,-0.034141 -0.1448474,0 -0.076341,0.076341 -0.08496,0.1998643 -0.1448474,0.2896948 -0.037876,0.056814 -0.1232548,0.08007 -0.1448474,0.1448474 -0.030537,0.09161 0.030537,0.1980853 0,0.2896948 -0.043185,0.1295555 -0.213943,0.1760672 -0.2896948,0.2896948 -0.059887,0.089831 -0.08496,0.1998643 -0.1448474,0.2896948 -0.037876,0.056814 -0.1143109,0.083774 -0.1448474,0.1448474 -0.021593,0.043185 0,0.096565 0,0.1448474 -0.048283,0.048283 -0.1143109,0.083774 -0.1448474,0.1448474 -0.021593,0.043185 0.034141,0.1107066 0,0.1448474 -0.034141,0.034141 -0.1232548,-0.043185 -0.1448474,0 -0.043185,0.08637 0.030537,0.1980853 0,0.2896948 -0.021593,0.064778 -0.1143109,0.083774 -0.1448474,0.1448474 -0.043185,0.08637 0.068282,0.2214131 0,0.28969482 -0.034141,0.0341409 -0.1107065,-0.0341409 -0.1448474,0 -0.034141,0.0341409 0,0.0965649 0,0.1448474 0,0.0482825 0,0.0965649 0,0.1448474 0,0.4345422 0,0.8690844 0,1.3036266 0,0.090693 -0.033333,0.80241788 0,0.86908438 0.030536,0.061073 0.1143109,0.083774 0.1448474,0.1448474 0.043185,0.08637 -0.043185,0.2033245 0,0.2896948 0.030537,0.061073 0.1232548,0.08007 0.1448474,0.1448474 0.043454,0.1303627 -0.043454,0.3041796 0,0.4345422 0.034141,0.1024226 0.1107065,0.1872722 0.1448474,0.2896948 0.030536,0.09161 -0.043185,0.2033245 0,0.2896948 0.030536,0.061073 0.1143109,0.083774 0.1448474,0.1448474 0.021593,0.043185 -0.034141,0.1107066 0,0.1448474 0.034141,0.034141 0.1107065,-0.034141 0.1448474,0 0.034141,0.034141 -0.021593,0.1016623 0,0.1448474 0.030537,0.061073 0.096565,0.096565 0.1448474,0.1448474 0.048283,0.048283 0.083774,0.1143109 0.1448474,0.1448474 0.043185,0.021593 0.1180651,-0.040173 0.1448474,0 0.084693,0.1270396 0.1078164,0.2864184 0.1448474,0.4345422 0.01171,0.046841 -0.043185,0.1232549 0,0.1448474 0.08637,0.043185 0.1950051,-0.018938 0.2896948,0 0.1497176,0.029943 0.2979788,0.076566 0.4345422,0.1448474 0.2477582,0.1238791 0.041937,0.3106631 0.2896948,0.4345422 0.1295554,0.064778 0.2971279,-0.045805 0.4345422,0 0.045805,0.015268 -0.043185,0.1232549 0,0.1448474 0.08637,0.043185 0.1980853,-0.030537 0.2896948,0 0.1295554,0.043185 0.1675487,0.2286218 0.2896948,0.2896948 0.08637,0.043185 0.2214131,-0.068282 0.2896948,0 0.034141,0.034141 -0.043185,0.1232549 0,0.1448474 0.1484787,0.074239 0.57491496,-0.074661 0.724237,0 0.061073,0.030537 0.0837744,0.1143109 0.1448474,0.1448474 0.0898886,0.044944 0.51495771,0 0.5793896,0 0.20716904,0 0.7186015,0.050161 0.8690844,0 0.10242258,-0.034141 0.1872722,-0.1107065 0.2896948,-0.1448474 0.1832191,-0.061073 0.3920261,0.046841 0.5793896,0 C 1.8044118,5.6783297 1.8262024,5.5879189 1.8830162,5.550043 2.1542787,5.3692014 2.3359907,5.4411899 2.6072532,5.2603482 2.8438191,5.1026376 2.666714,5.0456697 3.0417954,4.9706534 c 0.1420346,-0.028407 0.2940196,0.035131 0.4345422,0 C 3.5425806,4.9540927 3.5729025,4.8740885 3.621185,4.825806 3.7177499,4.7292411 3.7937776,4.6063725 3.9108798,4.5361112 4.041804,4.4575567 4.2088586,4.4595455 4.345422,4.3912638 4.406495,4.3607273 4.4334556,4.2842923 4.4902694,4.2464164 4.5801,4.1865294 4.6873868,4.1571155 4.7799642,4.101569 4.9292408,4.0120031 5.0588001,3.8897274 5.2145064,3.8118742 5.3510698,3.7435925 5.5269026,3.7586364 5.6490486,3.6670268 5.7354189,3.6022491 5.7175547,3.4536733 5.793896,3.377332 5.8702373,3.3009907 6.0072495,3.3088259 6.0835908,3.2324846 6.2846744,3.031401 6.6245186,1.9468578 6.6629804,1.6391632 c 0.023955,-0.1916385 -0.027313,-0.3882008 0,-0.5793896 0.2059204,-1.44144297 0.1448474,0.7600245 0.1448474,-1.01393178 0,-0.09656494 -0.043185,-0.2033245 0,-0.2896948 0.4180893,-0.83617862 0.2236664,0.33014192 0.4345422,-0.724237 0.018938,-0.0946897 0,-0.19312992 0,-0.28969482 0,-0.3379773 0,-0.6759545 0,-1.0139318 0,-0.1448474 0,-0.2896948 0,-0.4345422 0,-0.096565 0.035863,-0.2000365 0,-0.2896948 -0.064654,-0.1616338 -0.2250413,-0.2729084 -0.2896948,-0.4345422 -0.035863,-0.089658 0.018938,-0.1950051 0,-0.2896948 -0.029943,-0.1497176 -0.1078164,-0.2864183 -0.1448474,-0.4345422 -0.01171,-0.046841 0.015268,-0.099043 0,-0.1448474 C 6.6617671,-4.7377624 6.2844643,-4.9093739 5.9387434,-5.1686646 5.884118,-5.2096336 5.854969,-5.2829755 5.793896,-5.313512 c -0.043185,-0.021593 -0.1107065,0.034141 -0.1448474,0 C 4.6833993,-6.2791613 6.3732856,-4.9755347 5.2145064,-5.7480542 5.0136392,-5.8819657 4.8058211,-6.0118921 4.6351168,-6.1825964 4.4586951,-6.3590181 4.5672671,-6.5062161 4.345422,-6.6171386 4.2005746,-6.6654211 4.0557272,-6.7137035 3.9108798,-6.761986 3.8625968,-6.810269 3.8271058,-6.8762969 3.7660324,-6.9068334 c -0.043185,-0.021593 -0.1016622,0.021593 -0.1448474,0 -0.122146,-0.061073 -0.1601393,-0.2465096 -0.2896948,-0.2896948 -0.1832191,-0.061073 -0.3862597,0 -0.5793896,0 -0.4828247,0 -0.9656493,0 -1.448474,0 -0.048283,0 -0.096565,0 -0.1448474,0 -0.048283,0 -0.1107065,-0.034141 -0.1448474,0 -0.0341409,0.034141 0.034141,0.1107065 0,0.1448474 -0.0341409,0.034141 -0.11070654,-0.034141 -0.1448474,0 -0.0341409,0.034141 0.0215926,0.1016623 0,0.1448474 -0.0778531,0.1557063 -0.18524385,0.2952743 -0.2896948,0.4345422 0.014459,0.2869511 -0.21335352,0.068506 -0.2896948,0.1448474 -0.0341409,0.034141 0.0341409,0.1107065 0,0.1448474 -0.0341409,0.034141 -0.10166225,-0.021593 -0.1448474,0 C 0.08377437,-6.1520599 0.06107303,-6.0682855 0,-6.037749 c -0.04318515,0.021593 -0.11070654,-0.034141 -0.1448474,0 -0.0341409,0.034141 0.0341409,0.1107065 0,0.1448474 -0.0682817,0.068282 -0.22141308,-0.068282 -0.2896948,0 -0.0341409,0.034141 0.0341409,0.1107065 0,0.1448474 -1.3825e-4,1.383e-4 -0.43388569,-2.188e-4 -0.4345422,0 -0.0647777,0.021593 -0.0837744,0.1143109 -0.1448474,0.1448474 -0.043185,0.021593 -0.1107065,-0.034141 -0.1448474,0 -0.034141,0.034141 0.034141,0.1107065 0,0.1448474 -0.034141,0.034141 -0.1016623,-0.021593 -0.1448474,0 -0.061073,0.030537 -0.083774,0.1143109 -0.1448474,0.1448474 -0.043185,0.021593 -0.1107065,-0.034141 -0.1448474,0 -0.068282,0.068282 0.068282,0.2214131 0,0.2896948 -0.2286514,0.2286514 -0.088755,-0.3575027 -0.2896948,0.1448474 -0.073934,0.1848353 -0.096565,0.3862597 -0.1448474,0.5793896 -0.1448474,0.2414123 -0.3086369,0.4724265 -0.4345422,0.724237 -0.043185,0.08637 0.068282,0.2214131 0,0.2896948 -0.034141,0.034141 -0.1107065,-0.034141 -0.1448474,0 -0.034141,0.034141 0.021593,0.1016623 0,0.1448474 -0.030536,0.061073 -0.1143109,0.083774 -0.1448474,0.1448474 -0.021593,0.043185 0.034141,0.1107066 0,0.1448474 -0.1931299,0.1931299 -0.1448474,-0.2414123 -0.1448474,0.1448474",freehand=svg.append("path").attr("d",freehandCoord).style("fill","none").style("stroke","red").style("opacity",0).style("pointer-events","none"),xAxis=d3.axisBottom(xScale),yAxis=d3.axisLeft(yScale),gX=svg.append("g").attr("transform","translate(0,"+(h-padding[2])+")").call(xAxis),gY=svg.append("g").attr("transform","translate("+padding[3]+",0)").call(yAxis),xAxisText=svg.append("text").attr("x",padding[3]/2+(w-padding[1])/2).attr("text-anchor","middle").attr("y",h-padding[2]/2).text("Words per vote").style("font-family","sans-serif").style("font-size","12px");d3.timeout(function(){bars.on("mousemove",function(t,e){bars.style("opacity",function(t,a){return a===e?1:.25}),labels.style("opacity",function(t,a){return a===e?1:.25}),freehand.attr("transform","translate("+(xScale(t.words/t.votes)+10)+","+(yScale(t.site)+yScale.bandwidth()/2)+") scale(4)").style("opacity",1)}).on("mouseout",function(){labels.style("opacity",1),bars.style("opacity",1),freehand.style("opacity",0)})},3e3);
<script src="https://d3js.org/d3.v4.min.js"></script>

As we can see, supposing that this data is representative, one has to write almost 8x more words to get the same number of upvotes. I have to say that, for me, this is a reason for losing interest in participating at CR. Of course, we all know that the points system must be exactly the same in all SE sites, from SE Gardening and Landscaping to SE Worldbuilding.

So, this is my question: is there anything we can do? Or even better: should anything be done, or it's alright the way it is now?

• Just because people will upvote short low to medium quality answers on SO doesn't mean you should write such answers. Not every answer is going to get 26k upvotes like Mysticial's branch-prediction answer, but that's a good example of explaining something well. I also tend to write long answers that are more English than code, and have hit the 30k char limit in a few answers. If there isn't some interesting stuff to say it's probably a duplicate. – Peter Cordes Dec 26 '17 at 7:20
• Well, the S/B ratio on CR is much lower than the one on SO. That being said, good questions on SO contain example code and are more generic, whereas here on CR we usually have a specific software, which isn't written for a broader audience in mind. – Zeta Jan 2 '18 at 11:10
• @Zeta Thank you very much for creating this query! – Gerardo Furtado Jan 2 '18 at 11:51
• @GerardoFurtado I've accidentally destroyed it, it should now work. Here are graph versions for completeness: data.stackexchange.com/stackoverflow/query/… and data.stackexchange.com/codereview/query/… – Zeta Jan 2 '18 at 11:59
• Other interesting queries: the traffic (view-weighted length) created for answers on CR doesn't follow SO ones. SO traffic stems mostly from <5000bytes answers, whereas CR traffic seems a lot more widespread. But my SQL-Fu isn't powerful enough to analyse this more thoroughly. – Zeta Jan 2 '18 at 15:54

I think it's unfair to Code Review to view reputation/vote as the metric for success of an answer or the real impact of your answer. The vote/reputation is a metric for Stack Exchange, but I feel here we're creating a bigger value. We're teaching people in a lot of ways, helping them and us at the same time. I know you probably know all of that, but my point is this is not taken into account.

one has to write almost 8x more words to get the same number of upvotes.

No we write 8x more words to help them get better, explain things and make sure they leave the question better than when they ask.

This is more important here than on Stack Overflow. I learned so much from post here! When I'm learning a new language I browse the tag here to see what are the common mistakes, best practices and small details about the language that you will not see in tutorials.

My post is a bit all over the place for explanation, but I just want to illustrate that you can't just look at votes and based the value of your contribution based on that.

• whether you like it or not, reputation is the only thing motivating many people to contribute to stack exchange. in fact I think the ability to earn rep is one of the few things that makes it "fair." that said, I'm not sure it really needs to be fair at all. – I wrestled a bear once. Dec 27 '17 at 15:59
• If it's the only thing that motivate them to contribute to Stack Exchange, I don't mind! We get value from them for internet points. What I want them to know is that on Code Review, your participation is worth more than just internet points(in my eyes) if you do it right. – Marc-Andre Dec 28 '17 at 16:21

IMO the answering scheme on SO is where the problem is. People answer quickies and get their upvotes, and walk away thinking they've made the Internet a better place.

Allow me to make a slight generalization here.

What I'm seeing on SO, from monitoring a specific tag on the daily, is a TON of similar questions involving the same handful of bugs, over and over. I finally got my gold badge on SO, so I can now hammer dupes... but a lot of "helpful" answers people put up on SO essentially boil down to this:

Try this:

{some code snippet}


And they get votes. Sure, it helped the OP. Sure, it might help someone that bothers searching for a solution to their problem... but it says what, not why, and doesn't explain how.

After spending quite some time reviewing code on CR, I've noticed a strange side-effect in my SO answering: I don't post quickies anymore. I just can't help explaining what's going on, why the compiler is throwing such or such error, how it's solved, and how to avoid it in the future... and it's not rare that I'll also point out problems the OP didn't ask about, but that are glaring in their code and just waiting to be the subject of their next SO question.

As a result, I end up spending anywhere from 15 to 45 minutes on a SO answer, which isn't too far off from the amount of effort I could put in a decent CR answer. And when I walk away I know I've made the Internet a better place.

So my conclusion is, CR isn't the problem. SO's voting culture is. I don't upvote "Try this: {code dump}" answers on SO. If people want a quick copy-pasta solution instead of actually understanding what's going on, it's their loss, and IMO that's not what SO was made for.

The best SO answers are the ones where the author clearly put a lot of time and effort. These are the answers that should be getting the votes. If they aren't, it's not CR's fault - it's SO's voting culture that's broken.

As for "words per vote"... as others have said, that's not a useful metric, specifically because you can post "helpful" answers that don't explain anything on SO and get votes for it.

Give a man a fish, and you feed him for a day; show him how to catch fish, and you feed him for a lifetime.

If SO's voting culture didn't value copy-pasta solutions that don't teach anything, your metrics would be quite different.

• I totally agree on the SO's damn quickies part! However, the question is still unanswered: can we improve the reputation reward on this site? IMO, while we get a professional satisfaction when helping others with their code, we're still expecting some tokens of appreciation. Especially, when they are one or two clicks away: up vote/accept. – Igor Soloydenko Dec 22 '17 at 17:59
• @IgorSoloydenko oh, that's simple: by voting! Strive to keep on top of this page, and you'll be doing your part to fix the problem (hmm, admittedly I haven't been voting much lately) - you can't vote for others, but everybody likes earning rep, so the more we vote as a community (and that starts with 1 user going out of their way to reward reviewers), the more rep we generate on the site. – Mathieu Guindon Dec 22 '17 at 18:04
• I understand that, but I think, it's too abstract of the suggestion while it sound concrete. I mean, I do as a rule accept answers in all the questions. Problem is, somewhy 2 out of 6 total questions are still unanswered (is my code decent? is it horrible noone even want's to review it?) -- nothing to accept/upvote. The third question gets an answer I strongly disagree with. The rest are accepted... Similarly, I do actively upvote answers in [javascript] and [typescript] where I farm the most. But I have the feeling not everyone does it. – Igor Soloydenko Dec 22 '17 at 18:12
• Another frustrating thing is when a new member posts a question, gets his/her answer and never reacts to it in any possible way whatsoever. Not even a comment. In this scenario, the answer is only "chancing" upvotes from other members. I also noticed that there is huge unbalance between JS/TS partition and Python/C#/Java/C++. I think, the latter ones are getting much more rewards per answer on average. – Igor Soloydenko Dec 22 '17 at 18:15
• The last observation is the amount of answers per question. I think, low number of votes results in people not answering a question, as long as it already has at least one answer already. May be because of "what's the point, the effort is not worth very little attention my answer will get now, when there's some other answer already". This is really bad, because as you said, reviews very often complement each other, so they shouldn't be considered as competing. – Igor Soloydenko Dec 22 '17 at 18:17
• It boils down to community engagement. SO is so huge that you can get away with never voting at all, and still score big with your answers (kind of selfish, isn't it). On smaller SE sites, one user not voting, vs that same user voting 20, 30, 40 times a day makes a gigantic difference. Merely discussing voting on meta is often enough to spark a surge in voting activity. Fly-by askers don't typically vote very much, indeed. But the bulk of the votes needs to come from the regulars, not fly-by askers. Hang around in Code Review Chat, talk about it - regulars love this site, and will help you. – Mathieu Guindon Dec 22 '17 at 18:19
• I don't know. Withing this year I've answered quite a few questions, and my top answer has only got 8 votes. Makes me feel my reviews are crappy :[ I think what you're suggesting is the thing I used to hear a lot in Russia where I'm from: "start with yourself". As I said, I'm trying hard on this part. But may be, just may be, the problem has the systematic nature? And I'd try to interpret the OPs question as "can we change something in the system to invite more active voting" – Igor Soloydenko Dec 22 '17 at 18:24
• Please look at this specific example too. I don't understand it. There's a question with a single +1 to it (well, I just added one to it), while there're three answers already. How's that? Do respondents not know that it's okay to upvote the question itself? codereview.stackexchange.com/questions/183427/… – Igor Soloydenko Dec 22 '17 at 18:44
• "After spending quite some time reviewing code on CR, I've noticed a strange side-effect in my SO answering: I don't post quickies anymore"... That also happened to me, but to a less extent. I still do one-line answers sometimes, if they are "complete", which is possible for some specific cases. I also don't like "try this" (which I've made in the past). However, my pet peeve at SO is "this will do the trick"... I don't do "tricks", we're not magicians, we're programmers. – Gerardo Furtado Dec 22 '17 at 23:22
• I spend most of my time in CodeReview and almost no time on SO any more. I have even found that when I'm looking for answers, I'm often better off searching CR first exactly because of the detailed answers and full context. – Edward Dec 24 '17 at 14:21

I think you'd be interested in Mat's Mug's answer to "What is the motivation to spend time doing a code review?":

I monitor and answer Stack Overflow questions in the tag - a tiny little tag in SO-land: has 173,998 questions, has 73,666.

I'm tracking progress for my gold tag badge on SO.

That's 5.2 votes per answer, and a much more balanced progress (1000/200=5.0).

And so on CR it's likely that you get twice the upvotes per answer. (Still not scientific, but it's better than 7 questions)

I however disagree with you thinking words vs reputation gained are a good measure of whether it's worth answering. I spend about 30 minutes to 1 hour on most of my answers. Doesn't matter if it's one that has 30 words, or 1,000 words. Because not all of my time is spent writing. I spend time understanding the code and improving the code, both skills that come better the more I do it.

If you don't think spending however much time you do on your answers is worth it, then I'd encourage you to spend less time on your answers. IIRC we had a meta, that explicitly encouraged users to not try and improve everything. There was a time where three answers posted an answer to one question, and none of the answers overlapped, but all were improvements to the question.

Not only that but we allow iterative reviews. If you don't think you should write a long answer, but split your answer into two separate instances then you can hope the OP follows the advice in your first answer, and comes back to get the rest of your advice.

Bottom line, you don't have to spend 20 hours on an answer, stop when you think you've written enough.

• That's an excellent point and, looking at that meta question you linked, I must say that mine is almost a duplicate, and could even be closed as such. – Gerardo Furtado Dec 23 '17 at 2:17
• @Peilonrayz: I'm curious, how do you generate those graphs of your progress re the [vba] badge? I have managed to find the "home page" for the gold tag badge in my own language — codereview.stackexchange.com/help/badges/96/c — linked from codereview.stackexchange.com/help/… — linked from codereview.stackexchange.com/help/badges — but I can't figure out how to get StackExchange to show me my personal "progress" the way you did. Got a URL? – Quuxplusone Dec 26 '17 at 18:42
• @Quuxplusone I'm sure there's some confusion here. 1. They're not my progress, they're Mat's Mug's progress. 2. Go to your profile and your activity tab which will shows those graphs. 3. You may see your progress to your next privilege, which you can change by clicking one of the cogs. – Peilonrayz Dec 26 '17 at 19:45

I think the real question is whether there actually is a problem.

The answers are generally longer here, but I suspect the relative amount of high-quality posts is also higher. One-liner answers you can get away with on Stack Overflow are usually unacceptable here.

If there is a problem at all, I think it's high-quality posts not being rewarded enough. This can be (and sometimes is) in part fixed with a bounty.

On top of that there's the point that longer answers are not necessarily of higher quality than short posts. I tend to prefer longer answers though, and I know a couple of users who share that sentiment, so I applaud your efforts in this regard.

Thank you for keeping the quality of Code Review high.

• I totally agree with you: the quality here, specially the questions, is way higher compared to SO. I'm sick of "gimme teh codez" questions over there, which are quite rare here. – Gerardo Furtado Dec 22 '17 at 12:01

There are many aspects that contribute to smaller reputation gains on CR compared to SO.

1. The word-count-to-vote ratio is a somewhat useful metric, but when we try to assess something in numbers, we definitely should not rely on a single indicator. For example, we could compare...

2. the head count difference. For the last seven days there were only 2'289 users visited CodeReview, whereas StackOverflow was visited by 659'153 users. That is already ~2.25 orders of magnitude in difference!

Less eyes means less direct reputation effects. It also means less engagement which might spark conversation and more activity in terms of clicks/up votes. And that's just the registered users, which are only a part of visitors.

3. Speaking of the visitors, I also realized that the sites are being used in fundamentally different main ways.

How does one get to StackOverflow? They normally search for an answer to a very specific question. Often, users get there directly from a search engine, they don't necessarily use the SO's search line...

If same person is interested in a Code Review, I highly doubt he/she will even search for the question in Google. Instead, they'll come to CR.SE (if they are aware of it) and just post their question, because 99% of the time their code is unique enough. I'm intentionally dropping the custom-quicksearch-implementation question category since they are a minority.

I believe this leads to questions that have thousand of up votes on StackOverflow. People there are luckier in a sense they can "invest into an answer once" and then keep receiving their dividends over the years! Here's an exaggerated example of such answer -- Jon Skeet has got 89k+ of reputation from up votes along (unless I misunderstand how reputation works with up votes). Now, I'm not saying that on SE you can answer one question and get tons of rep over time. No, for that you need to be very lucky to answer the "right" question in the "right" time. Yet, if you answer 100 questions, you'll see once in a while that you get +1 here or there and these reputation change do accumulate quite well.

So, you can see the huge huge difference in ROI, because in CR.SE such investment is hardly even possible today. If somebody disagrees, ask them how often do they use the search bar in Code Review before posting their question.

4. UPDATE A graph of a function of active users, active voters, new question count, and average question votes would be a very interesting metric to look at.

5. Who knows what else? I'm very sure there are many more important aspects here...

So, these measures may help to address the points from above:

1. You probably can not do much about word-to-count metric. At least, I don't see anything obvious here. If I have something to say, I say it, as long as it's important enough and useful.

2. Things will not change dramatically as long as this is a small community.

Increase the head count -- bring the new users in (popularize & advertise?) and work on retention (be a welcoming community).

3. Provoke users to look into old questions. There are at least two built-in mechanisms already, but I'd argue their efficiency: a) Community user bumps the unanswered questions once in a while to the top of the page; b) there are "Linked" questions in the right hand side when you're inside a question, no idea how much attention it receives... If I am not mistaken, it is being populated with the questions that we refer to in our questions/answers, so we need to make sure we create this connections to older posts that are relevant to the context.

Just my 2 cents.

Oh, and make sure you up vote my answer! ;)

• "Oh, and make sure you up vote my answer! ;)" I just did, not because you asked, but because it's a good one! The number of users play an important role, but here is a curiosity: I'm a biology teacher, so I answer at SE Bio as well, which is a community way way smaller than SO... however, funnily enough, at SE Bio I get 50, 60, 70 upvotes per answer... I have more than 1700 answers at SO, but I never had anything even close to that! So, it's interesting how a small community like Bio SE, with so few users, has such a high upvote rate. – Gerardo Furtado Dec 22 '17 at 23:11
• @GerardoFurtado well, may be my number-game part of the answer is plane wrong, i don't know. May be I misunderstand how things work. For example, in SE Bio there could be less questions in total, therefore each question gets a higher chance of being visited over time, thus higher chance to get up voted. Or may be (much more likely), biologists are just nicer people, haha :) – Igor Soloydenko Dec 22 '17 at 23:15
• That's correct: the number of questions is so small that almost everyone sees every new question. Also, at Bio, there is less segmentation: for instance, I cannot answer a question here with a C#, Haskell, Java or C++ tag, which are languages I don't know. However, at Bio, everybody answers the different tags: zoology, botany, evolution, biochemistry etc... – Gerardo Furtado Dec 22 '17 at 23:17
• @GerardoFurtado the latter part is very similar. I personally focus on providing reviews for [typescript], [javascript], [c#] code (in this specific order), but [c], [c++] are in my "don't" list because I hate those. :) So, specialization does exist here too, it's absolutely natural. – Igor Soloydenko Dec 22 '17 at 23:19
• @GerardoFurtado the high votecount on biology may be related to the number of questions it is getting overall. IIRC it has quite a few questions less than codereview. The days when one could read all questions and answers on codereview of a day in a day are most probably over. That alone can dramatically reduce the number of voters on site. – Vogel612 Dec 23 '17 at 0:09
• @IgorSoloydenko I posted some queries as comments on the original post. Feel free to include them. – Zeta Jan 2 '18 at 15:55

There is one more answer to this question and since I often have controversial opinions I'm ok with downvotes. That's the price for non-mainstream thinking...

It has usually nothing to do with how big or good your answer is. If it's a simple question then you'll get a lot of votes - even from answers that resemble those on SO and with just a couple of sentences or lines of code.

Most of the time it doesn't matter how good your answer is. If you are answering questions about some topic or technology that no body knows or understands you are not going to get too many upvotes - or maybe even none. This is how it works. People don't usually vote for questions (or answers) they don't understand and cannot verify their correctnes - especially if it goes way beyond naming conventions and good practices.

Some people are very strict about it and if they don't know what you are talking about you are not get a vote. I'm not, I sometimes blindly upvote answers if I have the impression it contains some useful information even if I cannot verify everthing myself.

The area certainly is one of these topics that very few people understand. There are barely 34 questions tagged with it.

You can make the answers more valueable and verifiable if e.g. you add some links to the docs or quote the docs.

And sometimes you just get votes because you already have a few of them... and other people think your answer must be good because others already voted for you. The same applies to downvotes. If your counter shows -1 then you are most likely to get another downvote then an upvote. It sometimes all depends on what comes first - was it a 1 then you'll be upvoted, was it a -1 you're most likely to be downvoted.

It usualy works this way with hot-questions. The most upvoted answer is the one that has most upvotes before the question went hot even if other answers might be better.

• There is a positive feedback, that's sure: if you start with a upvote you most likely will get another upvote, if you start with a downvote you most likely will get another downvote. – Gerardo Furtado Dec 27 '17 at 7:26
• Also, what you said regarding the d3.js tag is true: that's the curse of low-popularity tags, that also happens in SO. What I try to do here is, besides linking/quoting the docs, provide a working example (snippet) of the proposed solution. – Gerardo Furtado Dec 27 '17 at 7:28
• Questions on conventions are actually horrible because they are very subjective. Same applies to tiny syntax details (similar to this one -- 136 views for something subjective is not making me happy, I mean if only each and every question was getting that much attention!). Yet those questions somewhy get the most attention. This is saddening and demotivating. – Igor Soloydenko Dec 28 '17 at 10:26
• rxjs has same problems as d3.js you mentioned... – Igor Soloydenko Dec 28 '17 at 10:29