-3
\$\begingroup\$

Over the last 2 months I have seen a significant proportion of questions closed, or marooned/abandoned by the threat of being closed. I understand the official figures are in region of 20-40% (about 25% "closed" plus a number of abandoned, unanswered question). The most common reason is:

  • Lacks concrete context
  • Code doesn't work - when it does, but there are challenges / questions

I acknowledge that in some cases this is appropriate, because there is "literally nothing there" or it's "completely broken", and the author cannot be persuaded to improve it. Yet often the code has simply been been "over sanitised" from its real version. The author of the question cannot post the "real code" because it's:

  • Too large or complex. More than 300 lines (we can debate the level) of code is bordering on unmanageable in a CR context. How many "real things" can you do in 300 lines?
  • Proprietary. The author is not comfortable or not allowed to post the real version. So it needs sanitising, which is hard, especially without losing most/all of the "context".

Yet at the same time I notice that there are a large number of questions which do not get challenged or closed, and which are, obviously or openly declared as:

  • Toy home projects
  • Academic Learning exercises, eg "trying to implement a linked list"
  • Code challenges

These questions may not technically fall foul of "lacking concrete context", or other rules as they are currently interpreted. Yet they are often contrived and devoid of any "real world context" and therefore do not stand up well to probing such as:

  • "why do you have to do it this way, why not take a different approach altogether", or simply
  • "why not just use, std::list, which does exactly that".

Their "academic" nature makes them "useful" only in the momentary sense, their "lack of commercial context" makes them employ techniques that would never be tolerated in a commercial project: Almost no one should be implementing std::list.

Obviously it is easier for the author of such questions to "provide full code context". Yet, in my opinion, these questions lack general context, by definition. Sufficient context cannot be provided, because "they arguably have none". They are a shell, based on a play or an idea. There is "no real project".

Often this code doesn't compile (ie doesn't "work" even at the basic level), but it's so easy to understand/fix that reviewers gloss over that.


The situation as I have described it above, bothers me for 2 reasons:

  • There is very little traffic and activity on CR. Apparently this has been static or even declining over many years. It's niche to be point of being questionably useful because "it has a tiny audience". It certainly has very few new questions and answers. About 40 per week, apparently across all languages and perhaps 5/week for a "medium" traffic language like C++?
  • The quality of the questions that get answered is very low, bordering on trivial. The repetitiveness of these trivial questions is very high.

The questions, which appear "based on real projects", but fall foul of some rule/guideline and get challenged/closed are often from new users. These questions mostly do not "get edited/improved" and those users tend not to come back with other questions (in 2 months I have not seen either). This mechanism tends to enhance the above developments.

I think CR has the potential to be a fantastically useful resource for all involved, but it falls short of its potential for these reasons.


Of course the rules exist for a reason. Mostly this reason seems to be underpinned by:

  • Protect a potential reviewer from answering a question which "is a moving target"

I can understand how that could be a problem, yet my impression is the approach to these rules is causing "lots of false positives", ie questions and users that could have been brought "into the fold" and made a useful contribution to the community, are instead eliminated.

The way in which reviewers' often appear to approach non-trivial questions appears to be:

  • Can I find a way in which this question violates a rule, and therefore move to challenge / close it?

Rather than:

  • Is this code in fact based on a real project? => make some allowances for the need to reduce/sanitise.
  • Could a meaningful answer be given which would enhance diversify & grow content & community?

This approach seems to minimise the quantity and complexity of questions which get answered, and amounts to a continuous drive in a direction of "low volume of trivial level of questions."


No Rules are perfect. They misfire. Lets use the metaphor of a spam filter. We get "false positives", ie questions "filtered" when they should not have been. If all rules are written to increase the "spam score" then false positives are very likely. There needs to be balance to achieve an acceptable number of false positives & negatives. Some rules should "reduce the spam score". eg, as mentioned above:

  • Is this a new user, making an apparently genuine attempt
  • Is this based on a real project but suffers from reduced context due to necessary sanitisation / size & complexity reduction

This type of "rule structure", and more importantly approach/application by the reviewers, is likely to achieve a better balance between false positives and negatives.


I have seen a previous meta which knocks on a similar door, and the highest ranked answer seems to be: "We can only deal with toy examples". I don't agree. I think this approach is a shame and part of the root cause of the low traffic and low quality. There are many "isolated professionals" who would hugely benefit from a CR style site, which tolerated their "interesting yet sanitised questions", which are far too complex and intricate to be dealt with by SO's "how to" questions.

Caveats:

  • I am not saying that is a new change, but am basing this on the last 2 months, since I have been watching closely. The fact that it may be "status quo" does not make it less relevant IMO.
  • I am basing this purely on questions tagged C++, as I have not been looking at other languages on CR. So I am not and cannot comment on whether the same pattern applies there.
| |
\$\endgroup\$
  • 1
    \$\begingroup\$ Your count of questions and answers is off, especially if you only use C and C++ as the basis. Most of the questions are other languages, python comes to mind as well as JavaScript. \$\endgroup\$ – pacmaninbw Jan 31 at 15:20
  • \$\begingroup\$ @pacmaninbw Which part is off? Do you have better numbers? I don;t have access to stats. happy to amend with better numbers if you have them, and you think it's useful to the discussion / or they are out by a long way? \$\endgroup\$ – Oliver Schönrock Jan 31 at 15:38
  • 1
    \$\begingroup\$ Read the first answer to your question, there are ways to get the statistics. You could ask how on the 2nd Monitor. \$\endgroup\$ – pacmaninbw Jan 31 at 15:41
  • \$\begingroup\$ I sometimes see questions that should be migrated to Stack Overflow but get closed as Not Working. What should be the preference there? Migrating, where the OP might get some help, or closing as off topic? \$\endgroup\$ – 1201ProgramAlarm Feb 1 at 0:48
  • \$\begingroup\$ @1201ProgramAlarm Yes that could be an option. Or make them "on topic" on CR if "too involved" for SO. The question is "how to recognise" and "how to do". Actually the initial challenge is much simpler. See the examples I gave in comments of answer below. Most of these just need to "not be reactively pushed away" because we don't immediately understand them, because they are non-trivial and perhaps presented somewhat imperfectly. We could capture another say 10% of "interesting / non trivial / non-homework style" questions that way. \$\endgroup\$ – Oliver Schönrock Feb 1 at 0:58
  • \$\begingroup\$ Let us continue this discussion in chat. \$\endgroup\$ – Simon Forsberg Feb 2 at 23:15
  • 2
    \$\begingroup\$ A lot of comments were around here, discussing stats, on-topicness and effort required to ask a question. Most of them were relevant to the question, but comments are a bad place to put the insights gained from the discussion. For those interested, the conversation is available at this transcript. I want to encourage the participants to boil down their realizations into answers. Thanks :) \$\endgroup\$ – Vogel612 Feb 3 at 14:33
6
\$\begingroup\$

You are bringing up multiple different suspects and there's many things I'd like to say, I will try to print down some of them here:

Asking questions

Asking a good question with code from a commercial project is hard and takes a lot of effort to make it fit this site. Only few people go through this effort to do that.

While we would love it if an asker would copy-paste their real code from their real project and ask about it here, with detailed explanations of what their code is doing, there are natural reasons why they cannot do this: The code is proprietary and they know themselves that they should not simply copy-paste the code. So naturally, they try to simplify it to remove details. As a reviewer I always ask why. I double-check every single decision that has been made in the code. If the code is simplified I am unable to do this kind of double-checking. This leads to me making wrong assumptions and giving advice that does not apply and nobody is happy. We have had way too many of these situations in the past which is why this rule exist.

I don't think there is a good solution to this besides trying to make an asker provide as much details as they can, and possibly come up with an example project (not example code) that shows what they are trying to accomplish. The important thing here is that the example project needs to look real.

Sure, there are some cases where some users might be a bit trigger-happy in closing questions. Everyone votes according to what they think is best for the site. As always, I can't put a finger on which questions I could accept or which I will not accept. It always depends and I won't try to come up with yet another rule about what may be okay. The best approach is to deal with each question as they come. If you think that a question is answerable and doesn't need to be closed, bring it up on meta and/or in chat.

Comments on questions

Asking the questions "Does this work?" and "Are you asking for how to implement something that you haven't already?" is intended to save us all time and possible headache. They are not meant to be rude in any way (at least not when I ask them).

There are many questions where it is unclear whether or not the code is working, or what the OP intends to get out from asking the question on Code Review. These clarifying questions are meant to avoid misunderstandings. If you can think of a way to post a good comment to clarify these situations without making them sound rude or threatening, please see Frequently Posted Comments

Answering questions

The quality of the questions that get answered is very low, bordering on trivial. The repetitiveness of these trivial questions is very high.

Answering more difficult questions takes longer time, and usually doesn't get a very high reward (upvotes) because it's not seen by many. Therefore, some users might steer away from more difficult questions.

There is a lot of things you can do here to help out.

  • Share the questions you find interesting (Facebook, Twitter, Stack Exchange Chat, anywhere)
  • Answer the questions you find interesting
  • Vote on answers to questions you find interesting

How to make life easier for everyone

For a long time we have had our six magical questions, what I think might be needed is to have a different checklist on how to write a good question. I wrote a guide myself but it might be in need of improvement or completion. Ideally, if all items on the checklist are checked, then it would be a high quality question that deserves an answer.

Note: We cannot in any way guarantee that a question will get an answer. See How can I make my question get more attention?

| |
\$\endgroup\$
5
\$\begingroup\$
  • Too large or complex. More than 300 lines (we can debate the level) of code is bordering on unmanageable in a CR context. How many "real things" can you do in 300 lines?

    No, we allow more than 300 lines of code. In fact we close questions without enough code and have had our post character limit increased to do so.

  • Proprietary. The author is not comfortable or not allowed to post the real version. So it needs sanitising, which is hard, especially without losing most/all of the "context".

    I find it hard to empathize with this stance. In my experience I haven't seen this as a problem. Maybe because I stay in the Python ecosystem which makes mocking things super-duper easy?

    Legally speaking I don't think anyone should want to post confidential code. Whilst this may sound extreme and like it just wouldn't happen, we have had it happen. And a post was deleted due to the user's actions. [citation needed] We have decided that it's safer to not allow copyright violations, to protect the user and the answerers.

    In my opinion, if a poster can't be arsed to spend an hour or so re-implementing the code for a related, but different task. Then why should we help them? It is clear that the poster is the one lacking the desire to get our help. Why should we, volunteers, bend backwards for someone that won't even produce an MVP or description?

  • these questions lack general context by definition.

    Unfortunately we have had a scope creep. Back in 2018 when we redesigned the close reasons, we titled one of the close reasons "Lacks concrete context". Later we, seemed to, conclude that context does not mean what we intended. However, afterwards our close reason changed to reinforce this undefined term.

    Due to how recent the change in the close reason was, we seem to have come to an agreement to leave it be and see what happens. We can assume an outcome, but witnessing it trumps speculation.

    I feel this is the core to the misunderstanding here. And that is our fault. It is unclear that context is referring only to the code. To many they interpret it as things that are not the code.

    I however don't agree with you. You can and should learn many things when programming. Whilst implementing something like std::list may seem dumb, the skills I have been taught by these things help me in 'commercial' things. It's not the destination that matters, it's the journey.

    Additionally due to the problem being one that many can face; because most challenges are free and accessible with an Internet connection only. This allows for more people to see the problem and learn from the answers when they Google for a solution to a problem. I think these questions help the most people, the 90% that don't ask or answer.

    For clarity on what the close reason should mean please see these two things, 1 and 2.

  • Often this code doesn't compile (i.e. doesn't "work" even at the basic level), but it's so easy to understand/fix that reviewers gloss over that.

    Please vote to close or flag these posts. I find that it does take a while for us to close off-topic posts.

    I know that I don't help in the queues much. However I would get vexed too often and too much if I did.

  • About 40 per week, apparently across all languages and perhaps 5/week for a "medium" traffic language like C++?

    Python alone has 40 questions a week, where C++ has had 10 this week. I think you are thinking we are smaller than we actually are.

    But what's wrong with the amount of questions we have at the moment? Why do we need to grow, what is wrong with not growing, or our users shrinking?

    The only negative feelings I've had for the site 'shrinking', is when users I've interacted with years leave the site. It's like losing a friend. But even if we're growing the same happens. That's life.

  • The quality of the questions that get answered is very low, bordering on trivial. The repetitiveness of these trivial questions is very high.

    What do you want us to do? We have a bar that I think is fairly low, but according to you not low enough for "Proprietary" code, but not high enough for everything else?

    Many new users post these trivial questions. What are we supposed to do, help the newcomer or follow our rules and improve site quality?

  • those users tend not to come back with other questions

    That's kinda how the Internet works. When I post a question on a site I'm not thinking "how can I join this community?" or "how can I exploit this free labor to my benefit?" I just want an answer, and then I'll go back into my way of life. Which doesn't involve the site I posted on.

    A user not coming back after 2 months isn't a sign that we're hostile. It's also not a sign we're not hostile. It's just how things are.

  • I have seen a previous meta which knocks on a similar door, and the highest ranked answer seems to be: "We can only deal with toy examples".

    No, it's that "toy examples" are the easiest questions to post, but not the most enjoyable.

    Challenge problems are more attractive for reviewers


I feel your post is jumping between two different opinions:

  • We're not friendly.
    We scare away users by trying to uphold the quality of the site. And you think this can be seen by our site not growing.

  • We're allowing the wrong type of question.
    You seem vexed that we seem to prioritize "toy home projects" and programming challenges. Where you'd prefer if these were treated like dirt, in favor for 'proper' questions.

    Unfortunately, to you, these are very easy to make on-topic. Where a personal project requires at least some thought on a description. I've requested descriptions from many users, and find users that post programming challenges come back with the needed description more often than users with their own project. We're not targeting anyone, or a specific group of questions. Or at least I'm not thinking "how can I make those pesky users that post their own projects have a bad day? *Votes to close the post* *laughs maliciously*"

I have been driven to the pessimistic opinion that any feedback, which isn't an answer, to new user is seen as power users being toxic, and trying to power trip. Where as a power user, all I want is the user to follow my advice and get their post on-topic so I can have more fun things to answer.

IMO the system is broken, much like you do. But clearly in different ways.
So, what's a specific problem, and how do we fix it?

| |
\$\endgroup\$
  • \$\begingroup\$ Thanks. I think, I may have overshot the mark on critiquing "academic/non-commercial" questions. Yes there is value in those, not I don't want them gone. I am disappointed that that is (nearly) all we have and I am wondering how to improve that. Because I believe we could "double" (??) the audience and usefulnes, if we did it well. re 300 lines: I know that's not the limit (hence the caveat in my question), yet in practice what do see? If there is more than 200 lines, people sigh. Yes the python/js/php culture on "open" code is quite different to c++ or even java. \$\endgroup\$ – Oliver Schönrock Jan 31 at 15:32
  • \$\begingroup\$ There is something else intriguing in your reply. On one hand: "Why should we, volunteers, bend backwards"..yet on the other: "so I can have more fun things to answer". I think the impression that the people writing questions "are the ones need help from the volunteer reviewers", that "question authors are the recipients of the advice produced by the reviewers" doesn't quite tell the full story. ..... \$\endgroup\$ – Oliver Schönrock Jan 31 at 15:36
  • \$\begingroup\$ What are you trying to say with this: "Legally speaking I don't think anyone should want to post confidential code."? I don't get it. Of course not?! They can't. It would be illegal? The point is they have to rewrite it. I have done that. It takes time. More time than to answer the reduced question. Many don't make that effort, or not properly. They should. We should encourage them to do that. That's all I am suggesting. Because it would create "more users" and "more interesting questions"... everyone wins? \$\endgroup\$ – Oliver Schönrock Jan 31 at 15:51
  • 2
    \$\begingroup\$ @OliverSchönrock I will try to answer you to the best of my ability, however, I find it hard to understand the subject of your pro clauses. As the subject seems to change without you being explicit. I feel like the majority of the answers to your questions can be summed up as "people follow the easy route". It'd be nicer if users posted more complex questions, and it'd be nicer for answerers to post more in-depth answers. But many factors, internal and external, encourage everyone to do the minimum. I don't think there's mal intent from any party, just human behavior. \$\endgroup\$ – Peilonrayz Jan 31 at 16:20
  • \$\begingroup\$ @OliverSchönrock I do think you speak of a utopia. How do you think we can get there? How do we encourage better questions or answers? How do we encourage users to write an MVP? I currently don't think it's possible for us to encourage someone that only sees the "Ask question" page before posting, to do all these things. But you may have insight on a way how. \$\endgroup\$ – Peilonrayz Jan 31 at 16:24
  • \$\begingroup\$ My fundamental suggestion is: when moderating / reviewing, to "bear in mind" that new users and their questions are "wanted" especially if they look like they might diversify the "homework questions" portfolio we have. I have written about an equal number of questions and answers so far. I have found the questions a struggle. Each time I was challenged, sometimes threatened to close the question etc etc. Sometimes I had "academic/trivial" style questions: these were easier to "get past" the reviewers. The "harder / more real" ones we tougher. Despite considerable effort on my part. \$\endgroup\$ – Oliver Schönrock Jan 31 at 16:29
  • \$\begingroup\$ I have seen other sub 500 rep users struggle in the same way. They are posting what is obviously a real question, but falling foul of (probably well intentioned) rule enforcement. So I am not "demanding we manufacture utopia", I am just suggesting that when a "less perfect utopia comes knocking", try not to, with the best possible intentions, slam the door in its face. Case in point: codereview.stackexchange.com/questions/236425/… I managed to get in there and produce a useful answer, before it got shut. \$\endgroup\$ – Oliver Schönrock Jan 31 at 16:32
  • \$\begingroup\$ So, I completely accept that this is not the intention, however my personal experience and the experience of observing other similar users shows me that: It is much easier to post "trivial homework questions". They are "less often challenged", "better understood", and have a better chance of "survival". So my point is: The spread of questions we have is to some extent a product of the process we put them through. I know that might be painful to accept, however, that is my experience. I don't expect you to believe me, but I thought I should have a go at articulating it. \$\endgroup\$ – Oliver Schönrock Jan 31 at 16:43
  • \$\begingroup\$ @OliverSchönrock To be clear, I don't think you are demanding a utopia. I merely want to understand you. I am sorry, but I lack the skills to understand what makes that question on or off topic. I know I can understand comments. It may be hard, but do you have an example of a hostile comment, and could you say what makes it hostile? \$\endgroup\$ – Peilonrayz Jan 31 at 16:43
  • \$\begingroup\$ It's not always easy to find good examples retrospectively because a lot of it is about comments and the comments get deleted a lot. This question from yesterday: codereview.stackexchange.com/questions/236293/… It's a bit off the beaten track. not THAT many people are good at linear algebra and the Eigen lib. It's clearly a real project and "interesting". The first comment was (now deleted): "Can you confirm the code works, because that is one our rules". No attempt by the reviewer to compile it. They just didn't understand it. \$\endgroup\$ – Oliver Schönrock Jan 31 at 16:50
  • \$\begingroup\$ This is my last question: codereview.stackexchange.com/questions/236073/… It wasn't a great question. But it was from a real project and I struggled to extract the meaningful part. The first comment (now deleted) was: "I haven't decided how to vote on closing this yet. " . the justification was that the post "mentioned undefined behaviour" (the ultimate problem in C++). The whole point of the post was to get working code reviewed and explore options of definitely avoiding UB while retaining convenience and performance. I am stubborn, so the Q survived... \$\endgroup\$ – Oliver Schönrock Jan 31 at 16:57
  • \$\begingroup\$ @OliverSchönrock I don't understand the code, but from the text alone I understand this. The question is written in the future tense; which makes the question sound like a request for code. Additionally the bolded sections, which weren't there originally; sound like a request for code. Personally as I don't know or have a C++ compiler I can't verify this easily. And so may ask a similar question. However I agree the comment seems non-nonsensical, as it's not asking if it's working as intended or alternately phrased "Are you asking for help implementing something you haven't already"? \$\endgroup\$ – Peilonrayz Jan 31 at 16:59
  • \$\begingroup\$ She (Lisa) was asking to review her use of value copies references / pointers in a high volume situation. She didn't ask it perfectly I agree. But the question is totally legit..these are very common and non-trivial C++ issues. I would not expect anyone who is not very experienced with C++ to understand that. The question is, did the reviewer who write that comment have the C++ experience to "see the real question". I did...I answered it. Lisa is happy. We got good content. I enjoyed it. All good. But a close shave. \$\endgroup\$ – Oliver Schönrock Jan 31 at 17:02
  • \$\begingroup\$ Found another one: codereview.stackexchange.com/questions/236400/… The first comment (now deleted -- spot the pattern) was "does this work?" (you can tell by the posters response). He made the poster doubt himself and change his code, which introduced a bug, which the poster then reverted. So back to sqaure1. Poster has had zero feedback on his code. No one has compiled it. Just been harrassed by 2 people I agree the question is not brilliantly phrased and there are issues with the code (missing includes etc), but still? \$\endgroup\$ – Oliver Schönrock Jan 31 at 17:22
  • \$\begingroup\$ Just as a contrast, this poster who wanted to know other ways of printing a string to std::cout had no trouble with being challenged and got lots of upvotes: codereview.stackexchange.com/questions/236434/… This is how the impression arises that maybe we are only challenging questions which we do not immediately understand because they are not utterly trivial? \$\endgroup\$ – Oliver Schönrock Jan 31 at 18:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .