What would you say is the biggest problem Code Review is facing as a site as of June 2015?

Question

Hopefully, by raising concerns, we may generate some ideas for improving the site!

Answer 1

Code Review is not graduated (Note: WIP)

This site has become the pity-site - the site that is graduated, but is not graduated, it's big, but not big enough, it's a beta site, but it's too busy to fit with the beta logic.

The biggest issue for Code Review is uncertainty. We can't attract more answerers because the people "in the know" don't look at Code Review because it is not graduated... but we need them to get us to the next level.

In a sense the site is the adult sitting at the kids table at thanksgiving, because that's the way it has always been....

If Code Review was a full fledged site it would rise in the esteem of the community it needs to attract to take the answering skills and knowledge to the next level.

Code Review needs to graduate in order to grow.

Answer 2

We need more answers and reviewers.

Our zombie rate is going up - fast, and the percentage of questions answers is falling. We need to attract more reviewers, and convert posters to reviewers.

Issues

When I first actively joined CR, I was afraid to post answers. I didn't know a whole lot about programming (I still don't - at least nothing like the big guns), and I was afraid I couldn't post a high-quality answer. I was told many times that as long as I had a justifiable comment, I could post an answer, which has drawn me in to answering more, and taught me a lot about programming. Maybe other users have the same issue I did, and are afraid to post answers?

Ideas for thought

One possibility to draw answerers in is to have a community-challenge style program designed to draw answers, not questions, in. When considering this, keep in mind that we must remember that this could potentially draw poor answers in.

Answer 3

Too much broken code

According to Question Close Stats (2K+ / Moderator Tools link), in the last 30 days we have closed:

• 225 questions with broken code - 58.4% of closed questions. 14 were reopened.
• 64 questions with hypothetical/pseudo-code - 16.6% of closed questions, 10 were reopened.
• 32 unclear questions (8.3%) - 11 were reopened.
• 29 questions that didn't include the code (7.5%) - 16 were reopened.

Questions Closed: 385 | Questions Asked: 1,208 | Close percentage: 31.87 %

Some days are worse - take today for example:

Questions Closed: 19 | Questions Asked: 47 | Close percentage: 40.43 %

---

It has been brought up recently on meta that we might be an elitist site that cherry-picks their questions (paraphrasing, I don't feel like searching for that link). I strongly believe we are not. If we had to pick only a single close reason to stay and every other one had to go, I would vote to keep the broken code close reason.

Thing is, that close reason is so predominantly used, that filtering it out completely changes the close reasons summary:

Past 30 days:

Questions Closed: 160 | Questions Asked: 1,208 | Close percentage: 13.25 %

Today (a pretty bad day - blame Monday):

Questions Closed: 11 | Questions Asked: 47 | Close percentage: 23.4 %

---

• We have the rulebook, but people don't read it.

  • Can it be reformatted to be more to-the-point? As in, putting MUCH more emphasis on the "BROKEN CODE IS OFF-TOPIC" part?

• We have a how to ask panel on the ask a question page:

...but people don't have their attention on that area of the page at that moment and it just appears to be invisible to most.

The only thing I can think of, would be a violent red popup that jumps at you when you try to ask something like..

Notice how blatantly off-topic this question is - and nothing in the UI is telling the user "Wait! If your code isn't working as expected, you're on the wrong site!" ...it actually suggests other questions that may have your answer.

I would love to see the percentage of our off-topic "broken code" questions that are asked by low-rep Stack Overflow users that were told to come post here.

I realize it would require some development effort, but having such a popup when the wording of the question title (and/or parts of the body?) makes it 95%+ certain that an off-topic question is about to be asked.... might cut our close rate by a little less than an optimistic half.

Answer 4

Not all tags have enough reviewers.

Objective C has @nhgrif answering most of the questions, and I personally find it hard to get ActionScript code reviewed (although it's practically Javascript with classes and strong typing).

Looking at the other answers, maybe the point is more this:

There's not enough answerers.

Code Review is pretty special here:

• Most questions don't deal with problems the average user would ever come across in their lives.
• Questions are highly specific in nature.
• Thus, answering serves mostly to help the asker, rather than the community at large.

Contrast other sites and you'll find that the more popular ones deal with solving more general problems - and providing a general fix or general advice.

Could it help to create more questions that are about general issues? Such questions are not example code; rather, take a small snippet that you have an issue with, and post it up for review. Questions like Find min of 3 numbers hardcoded, Searching an element in a sorted array, Finding repeating numbers in an array, Getting/setting default values from my App.config... These questions are our most popular questions, and they get that way because they get hit by search engines.

Answer 5

Educating SO users

I think an advent of recent months has been finding out that CR quite regularly gets recommended on Stack Overflow incorrectly. It's phenomenal that we now have a @Duga (thanks @Simon) to keep scanning comments so we can address accordingly. While it's good that SO users think of us, sometimes it is not appropriate.

What are some other ways we can educate SO users about what really belongs here?

Answer 6

'Low Quality' questions

I believe a common reason for questions to go unanswered is that they are not interesting. I see way too many questions without sufficient context, without a good explanation about what the code does (or how it does it).

These questions are typically good enough to not be down-voted much and/or closed. They are also bad enough to go unanswered for a long time.

Askers could do a better job at convincing answerers to spend about 30 minutes reviewing their code.

Answer 7

Missing Code Review Tools

When doing CR here on CR, everyone wastes time by

• having to copy the code piecewise
• having to split it rather than adding comments in between
• not being able to see the formatted text (e.g., in a new windows) when writing long texts

I only watch tag Java and (after filtering out boring questions) there's much to do, but when this site gets bigger, I won't do more reviews as they cost needlessly too much time.

The standard SO format is not good enough for CR.

Answer 8

Niche tags

Hosch250 already stated we need more answerers. We also need to have a better spread of specialists over all the languages out there.

This question about Factor has been around since '11. It's a good question, but it doesn't have answers. Why? We have nobody speaking Factor on CR. It's a niche tag with only 3 questions. TCL has two questions (both answered). Even languages which are often posted on SO are sometimes scarce on CR. For example: MATLAB

If people find only two questions in the language they're interested in, they won't post their code. If nobody posts their code, there won't be a community. Pimgd recently posted a question on Community Building which addresses this problem.

We need more spread.

Answer 9

We have a big problem of not having a variety of quality answers. I notice the same people answering, moderating, and commenting on questions quite frequently.

How can we attract more users with a deep skill set?

This brings me to another point; user retention. I see many users posting for the first time, often referred from Stack Overflow, and that's great! But how can we encourage these users to keep coming back? How can we encourage new users to not only join, but stay, and ultimately participate in helping this site turn into something truly great?

I think that Rolfl makes a great point when mentioning that we aren't

graduated as a site.

He is exactly right, but the bigger question here is; how can we get there?

Answer 10

Answering from my phone so a little bit short.

One of the biggest problem for me is that people who upvoted a question e.g with no answer won't come back to check if there is any answer.
I know that the CR format is different to SO because people usually don't have the same problem but I wish that 'upvoter' would recheck the question to also upvote a given answer if it is worth the vote.
An answerer would hopefully come back to CR if his/her answer of a question with 5 or more votes won't only get 1 vote.

Answer 11

I am a fairly new user on CR and here are some of my thoughts:

1) How do you post follow up code? The user updates their code following suggestions from an answer but they still have questions or follow up. What's the best way to do this? A new question?

2) Sometimes multiple answers can be right. If it's a review, different people can point out different things but only one answer can be marked as the answer. Now, you have to say "see so-and-so's answer above". So one user who put effort into giving an answer will get nothing for their effort (except for the satisfaction of helping someone out).

3) Today, there was a question by a new user on SO that was asking for a code review. Their question got multiple down votes immediately. A user recommended them to CR (which was appropriate). They posted on CR and, again, their question got down voted. This time because the title didn't ask for a code review and a few other nit picky things. IMO, this doesn't form a good first impression overall (for CR or SO).

4) Not sure if this is really in line with the question but a lot of what I have seen so far are a lot of users making the same kind of mistakes over and over. It would be nice if there were questions/answers we could point them to that discuss those things (instead of linking outside of SE).

5) Get rid of BETA. It implies it's not ready.

Answer 12

There's not enough questions

I follow the css tag and there's rarely more than one question a day (not counting broken code questions), if that. I kept an eye on database for a while, but that seems to move even slower. It's rather difficult to stay engaged when there's little to no activity in the areas I feel most capable of contributing to.

Answer 13

Code Review systematically prefers the least important code

There are already a large number of intrinsic hurdles before production code can get reviewed; it can be against company policy, paid developers are likely less open to criticisms and new coders are the only ones actually actively looking for help.

But then Code Review comes to stomp on the little fire we have:

• Code can't really be anonymized (else it's "theoretical"),
• Good code doesn't get reviewed as much,
• Complex code gets ignored,
• Votes further award "lowest common denominator" questions,
• Code must be "snippeted", which makes extracts from production code unrunnable.

Even the low volume of commentary on questions is likely to contribute - newbies often get immediate, simple answers (naming! comments!) but no such activity exists for most professional questions.

To be clear, it's not that I don't accept the toy questions or challenges. However, there is a downward spiral, in that the prevalence of exclusively toy questions enforces the notion that Code Review is for toy questions.

I don't know how we could change this, but it seems one would need a dedicated call to giving attention and good-quality answers to professional code. We should at least keep up the impression - like in-person peer review, the emphasis should be on immediate and hypothetical suggestions, not ground-up rewrites. Voters should appreciate this, and not leave such answers ignored. Perhaps even bounties could get involved - who knows?

(Let's not forget that "looks good" after a few clarifying questions is a valid review, but Code Review has almost no such answers.)

If we can't shake of this stigma now, we may never be able to.

Answer 14

I think part of the issue is also one of applicability. When I have a problem, I go to a search engine and enter my question, and inevitably, StackOverflow pops up with an answer. However, when I'm looking at some code with an eye to refactoring, my first thought isn't to look on Code Review to see how it was done by others. In fact, I'm not sure what I would search for in a lot of cases.

That said, I think this site is an incredible resource and that more people should be using and reading it. I wonder if it would be possible to do some sort of "marketing" to attract users? Other answers have pointed out that lots of people on StackOverflow recommend it, but they usually recommend it incorrectly. If we could keep (or better yet increase) the recommendation rate, but improve the number of useful recommendations, that would be ideal. I don't know how to do that off the top of my head, though.

Answer 15

Meta Meta Meta

I'm not sure if I'd say it's our biggest problem, but I do feel it's A problem that there's a lack of activity here on meta. Oh sure, the "regulars" from the chat room are mostly here, but rarely does "the silent majority" share their thoughts and opinions about the site here.

I'm not sure that there's really anything to be done about it. I'm really just taking it as an opportunity to call it out and invite anyone reading this who only passively interacts with meta to get involved. This is your site too. We want to hear from you too.

Answer 16

Not enough downvotes.

The only questions I see with a negative score are questions that are either on hold, closed, deleted, or accumulating close votes on their way to being put on hold.

Frequently, I'll find that my downvote is the only downvote on a post.

Often times, I feel that a post's score isn't a measure of its quality but rather a measure of its view count. More views = more votes, all of them up (and for those of us who make most of their posts in tags that get fewer views, this can become frustrating, but that's a different point).

Answer 17

Summary

In direct answer to the original question, it seems that CR is struggling to develop a sufficiently deep community to grow and be successful. I don't know that I can add much to what has already been said, as I think that the existing answers capture all the main reasons. I just want to present some thoughts and suggestions.

Q's/A's are very specific

user1118321's and Pimgd's have a good point. The answers to a code review largely only benefit the poster due to the highly specific nature of the question and answer. Even if a question did have the answer you were looking for, you would probably not spend the time understanding the question and go elsewhere.

Q's/A's beneficial for short time

Another issue that may not have been fully described is, that not only do answers have a used-by-date but so to do the questions. You ask a question today about your code, get it reviewed and in 3 months time, the code you wrote has been rewritten. Is it worth updating the answers? You certainly can't update the question as it would invalidate the answers. Posts on CR are inherently shorter lived.

Low incentives

Another thing that stands out to me from the existing answers, is that there seems to be little incentive for answerers to participate. I'm kind of jealous of some of the questions on SO like this one as an example. The poster asks a simple question about a language feature and then gets over 5000 votes (building up a huge amount of rep). And I know why, when I search for a simple answer to a question like that and Google gives me a SO question I know I'm going to get the answer succinctly.

I think one thing that could be worth looking at, is increasing the amount of rep artificially because earning it is hard on CR. And if it is hard to earn, then you probably aren't going to offer bounties. I think bounties would generate more interest. So maybe we could have some bounty fairies go around and add bounties to worthy questions. In the end its funny money to intice us to create a community where we can mutually achieve quality code review and all become better programmers.