17
\$\begingroup\$

Lately I've seen many questions pertaining to Project Euler and similar that have problems with time limit exceeded. And this prompted the question, are these questions on topic? I mean, the code, while it produces the correct result, can be argued to be broken as it doesn't solve the problem (due to taking too much time). In other words, it doesn't provide any answer and thus doesn't work.

While I enjoy reviewing the algorithms, I can't help but to feel that sometimes the poster might only be interested in getting their score on whatever programming challenge site it is and simply uses this site to "cheat" and get the correct algorithm. Which isn't the purpose of CR I think, or is it? I'm quite unsure here.

How do you feel about this? Should "Time Limit Exceeded" be considered off topic and referred to Programmers? Or should we consider to give to correct algorithms? Sometimes you learn from just seeing how it is done too.

\$\endgroup\$
  • \$\begingroup\$ Related: time-limit-exceeded tag info \$\endgroup\$ – rolfl Oct 4 '14 at 23:25
  • 1
    \$\begingroup\$ Supposing that you find such questions abusive, where would you draw the line? \$\endgroup\$ – 200_success Oct 4 '14 at 23:33
  • \$\begingroup\$ Also related (but really old now): Online Contest Questions \$\endgroup\$ – rolfl Oct 4 '14 at 23:34
  • \$\begingroup\$ This summarised my feelings: meta.codereview.stackexchange.com/a/437/36120 \$\endgroup\$ – Emily L. Oct 5 '14 at 8:29
  • \$\begingroup\$ What if there has been a lot of effort put into the question? Would that make it on-topic? (And if so, why?) \$\endgroup\$ – 200_success Oct 6 '14 at 20:24
  • 1
    \$\begingroup\$ @200_success That is a significant amount of effort and a nicely written question. But the question reads: "Why is that code much faster than this code?" and not "How do I get the answer that I need for Project Euler, without it taking 52 hours to compute?". The former I have absolutely no problem with but the second I do, well at least if it is masqueraded as a code review. \$\endgroup\$ – Emily L. Oct 11 '14 at 12:43
  • 1
    \$\begingroup\$ There are plenty of willing reviewers for these questions. The supply and demand exist, and it would be unfortunate to ban the questions altogether. At the same time, I acknowledge that some people find this class of questions uninteresting. That is my main motivation for creating the time-limit-exceeded tag — you can use it to filter out those questions from your front page view. \$\endgroup\$ – 200_success Oct 11 '14 at 15:36
  • 3
    \$\begingroup\$ @200_success And there are also plenty of reviewers willing to review broken code and to debug it. But those questions are considered off-topic. So I do not agree with your motivation there. \$\endgroup\$ – Emily L. Oct 16 '14 at 10:34
11
\$\begingroup\$

Performance problems are common in code presented in Code Review. The most valued reviews often point out ways to be more efficient.

Time limit exceeded (TLE) problems are performance problems where the performance is poor enough to cause an online judge to 'fail' the solution. Does the code produce the right result? Probably. Does it produce it in the right amount of time? No.

There are a few perspectives to have when looking at these problems:

Learning experience

Many of these judging systems are designed as tools to aid learning. Does posting a question to Code Review reduce the learning value of the question? No, and this is why:

  • a person whose code is failing the performance aspect of an online judge is not learning anything anyway. All they know is that there must be a better way, but the tool does not say what that way would be. They need some outside intervention to provide the key knowledge for producing a better solution.
  • the alternative to Code Review would be a Google search, and a 'get lucky'. This would teach them less than a code review would, because it does not explain why their code is slow, instead it just is a cheat for them anyway

At a point of crisis in the development of an algorithm, the right solution is to get a helpful hand and redirect, and an "education".

Cheating

You have a valid concern when you say: "While I enjoy reviewing the algorithms, I can't help but to feel that sometimes the poster might only be interested in getting their score on whatever programming challenge site it is and simply uses this site to "cheat" and get the correct algorithm."

I think this does happen, but this is not really a problem we should concern ourselves with too much. The only person who suffers here is the person asking the question with a 'cheat' in mind. Our answers here are just one of many places where the person could find the answers. Sure, our answers here will be another place where other cheaters can find the results, but, it is one of many, and only the cheaters lose value. The people who have tried hard, and are struggling to find the right algorithm, and are looking for that next pointer, those are the ones with the most to gain from a review.

Additionally, the person suffering a TLE problem are not likely going to be able to simply reinvent the right algorithm. They have to get it from somewhere. Sure, a good Code Review answer will point out the right algorithm, it does not mean it has to write the whole program as well. Pointing out the right algorithm would be a useful part of a review. Only write out the full working solution if it is something that makes the review easy, or is something fun that you want to do anyway.

Working Code

You say: "it doesn't provide any answer and thus doesn't work" and I counter that with no, the online judge failed to let it run long enough to produce the right answer.

I think there are a number of ways to look at this type of problem. On the one hand, the code fails the online judge, and that is the code's purpose, so the code just fails, and thus would be off-topic.

On the other hand, the code also works for the person who runs the code (just not for the judge). It works for them, and produces the right result for their input. They want it happening faster though. Still, to the best of their knowledge, it (will) work, if given the opportunity.

Rewrite Necessary

You are correct that the solution to these TLE problems is typically a complete rewrite of the core algorithm. The need to move from an 'obvious' implementation to a more obscure, technical, or mathematical approach.

Just because the code needs a rewrite does not mean that it is not ready for a review. It also does not mean you have to rewrite it. In many cases, a high-level review and a pointer to the right algorithm would be excellent feedback to the asker. An explanation of why their algorithm is slow is what they need, and a meaningful pointer to what the right solution is, would be much more of a learning experience than anything else available.

Conclusion

I believe these problems are just a special kind of performance problem, a problem where there's an objective 'judge' which fails code beyond a threshold.

The code is functional, and produces the right results (slowly).

The askers are normally quite frustrated when they ask, and need an insightful 'redirect' in a gentle way. Code Review is a good place for that.

Often the problems are interesting, and fun to work with.

You don't have to answer them all, just the ones that appeal to you, and maybe there's someone who finds this question appealing in turn.

Remember, Code Review should be a place that is fun to review things on too. If it does not look like fun, then don't do it.

\$\endgroup\$
  • 6
    \$\begingroup\$ I agree that performance aspects are an important part of a code review. But in the case of TLE and programming challenge type questions I feel that the question is more about the algorithm than the actual code, in these questions the actual implementation is most often irrelevant. And then it isn't a code review any more but rather a "help me with this algorithm" question masqueraded as a code review. I guess this is what grinds my gears. \$\endgroup\$ – Emily L. Oct 5 '14 at 16:39
  • \$\begingroup\$ Almost all questions on Code Review tagged performance require algorithm rewrites. Sometimes it is because it is just too slow to get things done in the day, other times it is because some online judge determined it was too slow. Regardless of whether an online judge made the determination or not, all performance questions inevitably require similar reviews - "Do things differently!". Is your concern with performance questions in general, or only those that are online-judged? \$\endgroup\$ – rolfl Oct 5 '14 at 17:05
  • \$\begingroup\$ Non-programming-challenge type performance problems are more about the code than their challenge siblings in my opinion as they are actually solving a real problem in probably a larger code base where the code quality matters as opposed to a challenge where only the correct time complexity and algorithm correctness matter. So yes only with those questions that pertain to programming challenges. \$\endgroup\$ – Emily L. Oct 5 '14 at 22:24
  • 3
    \$\begingroup\$ "Remember, Code Review should be a place that is fun to review things on too. If it does not look like fun, then don't do it." - Thank you for the reminder. ++ \$\endgroup\$ – RubberDuck Oct 7 '14 at 8:30
5
\$\begingroup\$

If people want to cheat, they will find a way to cheat. Simply googling for "Project Euler Problem xx solution" will give results that explain the best approach to any given problem.

Would considering these posts off-topic for Code Review help? Absolutely not. It would perhaps remove Code Review as a possible "cheating source", but Code Review being a "cheating source" is not a problem. The beauty of Code Review is that you get a more personalized answer. You get to hear what people feel that you need to hear.

On Code Review, in addition to getting help in finding out the correct algorithm, they can get suggestions about their coding style, variable naming, etc. A lot of the Project Euler questions I have seen here also get very helpful comments about such things.

There is more than one thing you can learn on Code Review.

\$\endgroup\$
  • 2
    \$\begingroup\$ It's not that I disagree with the answer, it just doesn't feel like this answers the question. \$\endgroup\$ – RubberDuck Oct 7 '14 at 8:31
  • 1
    \$\begingroup\$ It does, I came here to write just that. TLDR would be 'this is on topic, all parts are up for review' \$\endgroup\$ – konijn Oct 14 '14 at 15:42
  • 1
    \$\begingroup\$ Simply googling for "Project Euler Problem xx solution" will give results that explain the best approach to any given problem. - yup. Links to Code Review Q&A's! \$\endgroup\$ – Mathieu Guindon Jun 7 '16 at 23:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .