Over the last 2 months I have seen a significant proportion of questions closed, or marooned/abandoned by the threat of being closed. I understand the official figures are in region of 20-40% (about 25% "closed" plus a number of abandoned, unanswered question). The most common reason is:
- Lacks concrete context
- Code doesn't work - when it does, but there are challenges / questions
I acknowledge that in some cases this is appropriate, because there is "literally nothing there" or it's "completely broken", and the author cannot be persuaded to improve it. Yet often the code has simply been been "over sanitised" from its real version. The author of the question cannot post the "real code" because it's:
- Too large or complex. More than 300 lines (we can debate the level) of code is bordering on unmanageable in a CR context. How many "real things" can you do in 300 lines?
- Proprietary. The author is not comfortable or not allowed to post the real version. So it needs sanitising, which is hard, especially without losing most/all of the "context".
Yet at the same time I notice that there are a large number of questions which do not get challenged or closed, and which are, obviously or openly declared as:
- Toy home projects
- Academic Learning exercises, eg "trying to implement a linked list"
- Code challenges
These questions may not technically fall foul of "lacking concrete context", or other rules as they are currently interpreted. Yet they are often contrived and devoid of any "real world context" and therefore do not stand up well to probing such as:
- "why do you have to do it this way, why not take a different approach altogether", or simply
- "why not just use,
std::list, which does exactly that".
Their "academic" nature makes them "useful" only in the momentary sense, their "lack of commercial context" makes them employ techniques that would never be tolerated in a commercial project: Almost no one should be implementing std::list.
Obviously it is easier for the author of such questions to "provide full code context". Yet, in my opinion, these questions lack general context, by definition. Sufficient context cannot be provided, because "they arguably have none". They are a shell, based on a play or an idea. There is "no real project".
Often this code doesn't compile (ie doesn't "work" even at the basic level), but it's so easy to understand/fix that reviewers gloss over that.
The situation as I have described it above, bothers me for 2 reasons:
- There is very little traffic and activity on CR. Apparently this has been static or even declining over many years. It's niche to be point of being questionably useful because "it has a tiny audience". It certainly has very few new questions and answers. About 40 per week, apparently across all languages and perhaps 5/week for a "medium" traffic language like C++?
- The quality of the questions that get answered is very low, bordering on trivial. The repetitiveness of these trivial questions is very high.
The questions, which appear "based on real projects", but fall foul of some rule/guideline and get challenged/closed are often from new users. These questions mostly do not "get edited/improved" and those users tend not to come back with other questions (in 2 months I have not seen either). This mechanism tends to enhance the above developments.
I think CR has the potential to be a fantastically useful resource for all involved, but it falls short of its potential for these reasons.
Of course the rules exist for a reason. Mostly this reason seems to be underpinned by:
- Protect a potential reviewer from answering a question which "is a moving target"
I can understand how that could be a problem, yet my impression is the approach to these rules is causing "lots of false positives", ie questions and users that could have been brought "into the fold" and made a useful contribution to the community, are instead eliminated.
The way in which reviewers' often appear to approach non-trivial questions appears to be:
- Can I find a way in which this question violates a rule, and therefore move to challenge / close it?
Rather than:
- Is this code in fact based on a real project? => make some allowances for the need to reduce/sanitise.
- Could a meaningful answer be given which would enhance diversify & grow content & community?
This approach seems to minimise the quantity and complexity of questions which get answered, and amounts to a continuous drive in a direction of "low volume of trivial level of questions."
No Rules are perfect. They misfire. Lets use the metaphor of a spam filter. We get "false positives", ie questions "filtered" when they should not have been. If all rules are written to increase the "spam score" then false positives are very likely. There needs to be balance to achieve an acceptable number of false positives & negatives. Some rules should "reduce the spam score". eg, as mentioned above:
- Is this a new user, making an apparently genuine attempt
- Is this based on a real project but suffers from reduced context due to necessary sanitisation / size & complexity reduction
This type of "rule structure", and more importantly approach/application by the reviewers, is likely to achieve a better balance between false positives and negatives.
I have seen a previous meta which knocks on a similar door, and the highest ranked answer seems to be: "We can only deal with toy examples". I don't agree. I think this approach is a shame and part of the root cause of the low traffic and low quality. There are many "isolated professionals" who would hugely benefit from a CR style site, which tolerated their "interesting yet sanitised questions", which are far too complex and intricate to be dealt with by SO's "how to" questions.
Caveats:
- I am not saying that is a new change, but am basing this on the last 2 months, since I have been watching closely. The fact that it may be "status quo" does not make it less relevant IMO.
- I am basing this purely on questions tagged C++, as I have not been looking at other languages on CR. So I am not and cannot comment on whether the same pattern applies there.
