32
\$\begingroup\$

November 15, 2013. 6:30AM UTC. Community Manager Grace Note posted How is Code Review doing right now?.

We are 100+ days later.


Rewind

Since mid-November 2013, a number of things have happened:

  • A tag was created on Meta-CR.
  • CR users answered the Call of Duty; incoming, as well as mummified and even some fossilized zombies keep getting shot at every day.
  • marked each weekend of at least the first half of December.
  • Winter Bash 2013 was a very successful event that marked the rest of December.
  • Over 5.7K stars were spent on over 3,300 starred posts in The 2nd Monitor, the name that was given to our site's main chatroom.
  • Stack Exchange Data Explorer made beta sites' data available for some serious querying.
  • A picture is worth 1000 words:

Site Stats

(2014-02-25 2:30AM UTC)


Let's look at the past 15 weeks:

Let's take Stack Overflow as a reference for activity:

SO Activity, 60 weeks

This is ours:

CR Activity, 60 weeks

Zoom in on the past 15 weeks:

CR Activity graph since Nov.16

  • Question and Answer voting immediately went straight up, and dipped for the Holidays, but never went anywhere near below where it was.
  • Two users racing for a [badge:electorate] has had a visible impact on the Question votes, but since a monkey won the race on Jan.27, Question votes have gone down a bit, but remains above 1K.
  • New users is on a steadily ascending slope.

Voting and Reputation Issues

One of the key points of our review was the voting issues:

Voting is an odd beast here. When we reviewed the site, the front page had a dearth of voting. Even looking now, in the last 100 questions asked and the last 100 answers posted, less than 10 of each have scores past 3 with several negatively rated. Now, there are users who vote - vote a lot in fact, as most of the top users have hundreds of votes under their belts (with decent downvote rates as well). But in spite of this, there’s still a humongous bed of users sitting in the ~100-200 reputation range and not actually in the major privilege levels. So for where-ever all the votes are going, it ain’t accumulating.

At the time of this writing, the last 100 questions asked on SEDE have these scores:

  • 50 have a score greater than or equal to 3.
  • 1 has double-digits upvotes.
  • 2 have a negative score.

Last 100 answers posted:

  • 42 have a score greater than or equal to 3.
  • 3 have double-digits upvotes.
  • 1 has a negative score.

I realize this is out-of-sync, ..but we're definitely in another ball park. Let's look at reputation scores.

The inner ring represents the distribution of reputation scores of avid users around November 25, 2013; the inner-middle ring represents the same figures as of Jan.16; the outer-middle ring represents the same figures as of Feb.16, and the outer ring is as of Mar.16:

enter image description here

In this conversation, Grace Note noted that "For separate percentages, you have 51% sitting in 150-200, 38% in 200-500, and 6% in 500-1000. That leaves 5% for everyone above 1000."

Page 1 of users page, reputation tab, all-time, ranges between 19.6K and 3.2K. Page 2 is still above 1.5K, page 3 is still above 1K; the distribution of active high-reputation users is starting to look like something. Voters? We have 2 pages for the month.

Time to Answer

In mid-November, the average time for a question to receive its first answer was somewhere between 8 and 10 hours.

Following the CodeReview Zombie meme, if you divide the answers on CodeReview in to two groups:

  1. Answers for questions that were younger than a day old (when answered)
  2. Answers for questions that were older than a day old (when answered)

The second group are a loose definition of Zombie Kills. There are two sides to our progress...

  1. keeping on top of new questions coming in
  2. clearing out the backlog of old unanswered questions.

This graph shows the activity in these two areas, :

Quick Zombies

It shows the nice red line, which is the number of questions per week that get answered within a day.

The 10's of zombies is the curve that shows the steady increase in unanswered questions, until mid November, when the zombie-cull commenced.

These two metrics clearly show how the zombies are being brought back under control, while still maintaining quality answers in good time for the new questions coming in.

Unanswered

One of the most pressing issues with the site was the gigantic pool of unanswered questions. CR was then at around 88% answered, with > 1K questions with no upvoted answers. At the time of this writing, CR is at around 95% answered, with 663 unanswered and 673 with no upvoted answers.


What's the question, you'll ask?

Are we there, yet?

\$\endgroup\$
  • 2
    \$\begingroup\$ I would like to point out that the monkey cheated in the race to get the Electorate badge! :P \$\endgroup\$ – syb0rg Feb 26 '14 at 4:51
  • 2
    \$\begingroup\$ Time to think of a great design for the final release. \$\endgroup\$ – Morwenn Feb 26 '14 at 14:24
  • 8
    \$\begingroup\$ +1 Great post, and reflective of just the kind of engagement that's driving a lot of the successes you're seeing above. Keep it up!! \$\endgroup\$ – Jaydles Feb 26 '14 at 19:45
  • \$\begingroup\$ Could we have an update of some sort ? What's our situation now ? \$\endgroup\$ – Marc-Andre May 23 '14 at 17:17
  • 1
    \$\begingroup\$ @Marc-Andre current situation is that we're still in beta... kidding, you're right, it's time for a quarterly update. I'll gather some numbers tonight :) \$\endgroup\$ – Mathieu Guindon May 23 '14 at 17:26
17
\$\begingroup\$

A lot of progress has been made, but we're not there, yet.

In the one month since we've had access to the data, we've barely made a dent in the distribution of avid users among reputation tiers (the "Grace Note query").

There is still a massive backlog of users who are still suffering from the historical effects of low voter participation. Even with the recent voting campaign, we seem to be promoting as many users into the ≥ 150-point tier as we are graduating into the upper tiers.

We're excited to have increased the ranks of our ≥ 10k users to six members. While we're focused on that figure, though, keep in mind that under the standard point scale for graduated sites, 10k points only grants access to moderator tools. It takes 15k points to be able to protect questions, and 20k points to be a full trusted user. (We're getting extremely close to having our first 20k user!)

Despite the effort to increase voting, I'm afraid that much of the voting is done by a dedicated group of supporters. Before we say that we're there, I'd like to see evidence of deep cultural change in voting patterns. (There's probably a Data Explorer query coming up to track the voting patterns!)

\$\endgroup\$
  • 4
    \$\begingroup\$ If only there was a way to track votes back to people, oh, wait, that would be no good ... ;-) \$\endgroup\$ – rolfl Feb 26 '14 at 19:45
  • \$\begingroup\$ @rolfl I assumed that since the aggregate number of votes cast is public information, that similar information would also be available through Data Explorer. Apparently not. The Voters list might be good enough to illustrate my point. \$\endgroup\$ – 200_success Feb 26 '14 at 19:51
  • \$\begingroup\$ Voting is great, but should one only vote on answer/questions where they are knowledgeable enough to know that it is good or not-so-good? As in, topics where they don't really know the language/platform, or voting for the sake of having votes? \$\endgroup\$ – Phrancis May 24 '14 at 1:37
13
\$\begingroup\$

Another noteworthy event was Best of Code Review 2013. Over the course of two weeks, the Code Review community found some great questions and answers that illustrate the benefits of this site. Hopefully it will become an annual tradition.

\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .