Reviewing AI based on how it performs

Question

On Code Golf is is common to see a king-of-the-hill question where you have to implement an AI based on the rules given in the question.

So then I got to thinking - "If I developed an AI for one of these 'games', I could post it on Code Review to improve it!" But then I was wondering how an AI would actually be reviewed here.

See, it's great to have my software run fast, not have any bugs, conform to the standards, etc. But when it is all said and done, I really just want my AI to outperform the other AIs. But in order to have that happen, a complete rewrite of my 'bot may have to be done (in cases where it turns out to be nothing like my code began with).

Would asking for a review like that be considered off-topic?

Answer 1

I want to agree with Simon's answer, however, I can't.

CodeReview clearly isn't "not primarily for modifying what your code does but rather how it does it."

Presuming the AI already accomplishes a specific task, I'm not sure how making that AI accomplishes that specific task better is outside the scope of Code Review.

We have the new time-limit-exceeded tag, which is intended to be used for code that works, but simply does not work fast enough. Users are allowed to ask reviewers to make the code accomplish the same task in a shorter time period, and this sometimes means completely rewriting the entirety of the code to implement a wholly different algorithm.

If I can say "This code finds all the primes under X, but it takes 7 minutes where it's supposed to work in under 1 minute" and be perfectly on topic for code review, then why can't I ask questions such as these and still be on-topic?

• This chess AI defeats me in 47 moves on average. I'd like for it to beat me in 25 moves.
• This Minesweeper AI solves boards in 93 seconds. I'd like for it to solve boards in 10 seconds or less.

The latter is clearly acceptable--it's just a different form of a time-limit-exceeded question essentially.

I'd like to make it clear that I think AI that doesn't already work is off-topic, but this is off-topic because it is broken code, not because it is a code asking about improving AI.

Beyond this though, if you have an AI that needs logic improvements, you could probably post pseudocode to GameDev.SE--heck, you might even be able to post the code itself, but pseudocode might be a better bet as it's not strictly a programming site.

Answer 2

Your prototype question:

See, it's great to have my software run fast, not have any bugs, conform to the standards, etc. But when it is all said and done, I really just want my AI to outperform the other AIs.

has two parts -

1. how do I write an AI that is a winning AI?
2. Is my current AI a good AI, and is it implemented well?

If you were to separate those parts in to different questions on Code Review, the first would be clearly off-topic. The second would be clearly on-topic.

But when you put them together in one question, the first part is actually casting doubt on the implementation of the second. You are saying "I will completely rewrite the current code if there is a better way to write the AI". You have declared that you have 'priorities'. Your priority is to first have a winning solution, and your secondary concern is a clean and fast solution.

As a result of your priorities, an answer that only addresses the clean & fast component of the current code will be of little value to you. An answer that addresses the 'winning' AI component though would be of much higher value to you.

Additionally, the way you have phrased your question (I really just want my AI to outperform the other AIs) is that you are already concerned that your solution is not optimal, and what you actually want is someone to recommend an alternate algorithm to implement in your AI.

You want a 'whiteboard' discussion of what your algorithm does, and how the algorithm can be improved. You want a critique on how you solved the problem first, and then, as a secondary concern, you want a critique on your implementation of that solution.

As a consequence, your question is off-topic, and should be closed

On Code Review, you are expected to have working code. Even though your program produces results, your are not sure that the results are the results you want. Your code is not ready for a peer review on Code Review, and you need to be satisfied that your code is working to the best of your knowledge (and abilities), before presenting it here.

When the code produces the results you want, then the question you will ask on Code Review is just the second question above:

• Is my current AI a good AI, and is it implemented well?

There is a subtle difference between:

• How do I write a winning AI?
• Is my current AI a good AI, and is it implemented well?

The difference is the shift in priorities and assumptions. In the first instance, a review of how you solve the problem is more important than how you implement the solution. In the second question, the assumption is that the way you have solved the problem is 'right', and the review of how you implemented the solution is more important to you (and an answer that addresses just the "is it implemented well" will be of high value to you).

I would suggest that it is implied on Code Review, that all questions have an implicit "is my solution good?" even when that is not specifically asked in the actual question. In other words, all Code Review questions start with:

Is my solution good, and is it implemented well?

This is what my code is supposed to do {...}

This is how I implemented it {...}

When you ask your question that way, there may in fact be some answers that come up and say: great code, but if you tweak your algorithm a little here, and there, you could do better!

So, an answer on Code Review may review any and all aspects of your code, including the 'design', but to the best of your knowledge, the code has to be producing the right (best possible) results before it is ready for review here on Code Review.

Answer 3

Code Review is not primarily for modifying what your code does but rather how it does it.

That being said, an answer can embed a feature-request inside a review this has been done before. An answer may after all comment about about any or all facets of the code ^{on-Topic question #6}.

If you explicitly asked for a review on How can I make the AI stronger? I would like to see that post edited to How can I improve my code? and a comment added:

Welcome to Code Review! Changing what your code does is not the goal of this site. Code Review is about doing the same thing in a cleaner way. Because of this I have edited your question to ask for how the code can be improved. If you are lucky though, a reviewer may suggest an improvement to the AI itself.