100 days later: what's up?

Question

November 15, 2013. 6:30AM UTC. Community Manager Grace Note posted How is Code Review doing right now?.

We are 100+ days later.

---

Rewind

Since mid-November 2013, a number of things have happened:

• A fun tag was created on Meta-CR.
• CR users answered the Call of Duty; incoming, as well as mummified and even some fossilized zombies keep getting shot at every day.
• weekend-challenge marked each weekend of at least the first half of December.
• Winter Bash 2013 was a very successful event that marked the rest of December.
• Over 5.7K stars were spent on over 3,300 starred posts in The 2nd Monitor, the name that was given to our site's main chatroom.
• Stack Exchange Data Explorer made beta sites' data available for some serious querying.
• A picture is worth 1000 words:

_{(2014-02-25 2:30AM UTC)}

---

Let's look at the past 15 weeks:

Let's take Stack Overflow as a reference for activity:

This is ours:

Zoom in on the past 15 weeks:

• Question and Answer voting immediately went straight up, and dipped for the Holidays, but never went anywhere near below where it was.
• Two users racing for a [badge:electorate] has had a visible impact on the Question votes, but since a monkey won the race on Jan.27, Question votes have gone down a bit, but remains above 1K.
• New users is on a steadily ascending slope.

---

Voting and Reputation Issues

One of the key points of our review was the voting issues:

Voting is an odd beast here. When we reviewed the site, the front page had a dearth of voting. Even looking now, in the last 100 questions asked and the last 100 answers posted, less than 10 of each have scores past 3 with several negatively rated. Now, there are users who vote - vote a lot in fact, as most of the top users have hundreds of votes under their belts (with decent downvote rates as well). But in spite of this, there’s still a humongous bed of users sitting in the ~100-200 reputation range and not actually in the major privilege levels. So for where-ever all the votes are going, it ain’t accumulating.

At the time of this writing, the last 100 questions asked on SEDE have these scores:

• 50 have a score greater than or equal to 3.
• 1 has double-digits upvotes.
• 2 have a negative score.

Last 100 answers posted:

• 42 have a score greater than or equal to 3.
• 3 have double-digits upvotes.
• 1 has a negative score.

I realize this is out-of-sync, ..but we're definitely in another ball park. Let's look at reputation scores.

The inner ring represents the distribution of reputation scores of avid users around November 25, 2013; the inner-middle ring represents the same figures as of Jan.16; the outer-middle ring represents the same figures as of Feb.16, and the outer ring is as of Mar.16:

In this conversation, Grace Note noted that "For separate percentages, you have 51% sitting in 150-200, 38% in 200-500, and 6% in 500-1000. That leaves 5% for everyone above 1000."

Page 1 of users page, reputation tab, all-time, ranges between 19.6K and 3.2K. Page 2 is still above 1.5K, page 3 is still above 1K; the distribution of active high-reputation users is starting to look like something. Voters? We have 2 pages for the month.

Time to Answer

In mid-November, the average time for a question to receive its first answer was somewhere between 8 and 10 hours.

Following the CodeReview Zombie meme, if you divide the answers on CodeReview in to two groups:

1. Answers for questions that were younger than a day old (when answered)
2. Answers for questions that were older than a day old (when answered)

The second group are a loose definition of Zombie Kills. There are two sides to our progress...

1. keeping on top of new questions coming in
2. clearing out the backlog of old unanswered questions.

This graph shows the activity in these two areas, :

It shows the nice red line, which is the number of questions per week that get answered within a day.

The 10's of zombies is the curve that shows the steady increase in unanswered questions, until mid November, when the zombie-cull commenced.

These two metrics clearly show how the zombies are being brought back under control, while still maintaining quality answers in good time for the new questions coming in.

Unanswered

One of the most pressing issues with the site was the gigantic pool of unanswered questions. CR was then at around 88% answered, with > 1K questions with no upvoted answers. At the time of this writing, CR is at around 95% answered, with 663 unanswered and 673 with no upvoted answers.

---

What's the question, you'll ask?

Are we there, yet?

Answer 1

A lot of progress has been made, but we're not there, yet.

In the one month since we've had access to the data, we've barely made a dent in the distribution of avid users among reputation tiers (the "Grace Note query").

There is still a massive backlog of users who are still suffering from the historical effects of low voter participation. Even with the recent voting campaign, we seem to be promoting as many users into the ≥ 150-point tier as we are graduating into the upper tiers.

We're excited to have increased the ranks of our ≥ 10k users to six members. While we're focused on that figure, though, keep in mind that under the standard point scale for graduated sites, 10k points only grants access to moderator tools. It takes 15k points to be able to protect questions, and 20k points to be a full trusted user. (We're getting extremely close to having our first 20k user!)

Despite the effort to increase voting, I'm afraid that much of the voting is done by a dedicated group of supporters. Before we say that we're there, I'd like to see evidence of deep cultural change in voting patterns. (There's probably a Data Explorer query coming up to track the voting patterns!)

Answer 2

Another noteworthy event was Best of Code Review 2013. Over the course of two weeks, the Code Review community found some great questions and answers that illustrate the benefits of this site. Hopefully it will become an annual tradition.