What Site Policy should we have for code written by ChatGPT?

Question

As all of us I'm sure are aware, ChatGPT has induced a plethora of discussion network-wide. Specifically with this site, we could very well see an influx of questions about reviewing ChatGPT generated code (i.e, homework assignments askers want us to review before submitting, etc). This code would clearly violate one of our principal rules: "You cannot submit code to be reviewed that you did not write".

What, if any, policy have the Code Review Moderators planned or discussed to combat this potential issue?

Answer 1

As you say, that would be posting code not written by the asker. However, we do allow code that is maintained by the asker, and it would seem to fall into that category. That's not quite enough, as the following arguments show.

The clear reason to close such questions is that one cannot post anything on Stack Exchange that is in violation of copyright. Since such code is not licensable under the Creative Commons licence used here, it should be removed immediately.

As a guide for future policy (should the licensing terms change, for example), I feel that reviewing generated code has very little benefit for anyone - certainly not the poster, if they are unwilling/unable to write their own code in the first place! Remember that askers need to understand the code and how it works - without that, they are not able to provide adequate code context or to benefit from the reviews (i.e. incorporating suggested changes).

In any case, I feel this is just a special case of Reviewing generated code, so many of the responses there are relevant. (Did you write the generator? Are we reviewing the generator itself? Have you shown the input you provided to the code generator?)

Answer 2

Update:

Since February '24, the policy has been more explicitly written out in the help center.

---

A growing number of users are posting content that violates policy by including ChatGPT generated content. Warnings and suspensions have been handed out. Details about how we catch and/or handle these posts are scarcely made public to avoid people 'gaming the system'. If you see content you believe to be generated by ChatGPT (or similar), feel free to raise a moderator flag on the post indicating your concerns. Any obvious flaws in the post can be left in the comments, as with any post.

---

Toby already wrote a good answer. Since the question is about policy from the moderators, as a moderator, I'll be clear about this:

ChatGPT-generated questions and answers are not accepted.

ChatGPT-generated code is not accepted.

There are multiple reasons for this, but the most obvious one is plagiarism.

The licensing involved with posting a question to Code Review, or anywhere else on the Stack Exchange platform, does not work well with content that's generated based on thousands of other sources. It is unclear what the original licensing of those sources was and it can not be verified afterwards. In this regard, ChatGPT content is much more problematic than generators that at least disclose their dataset and part of their process.

We've seen questions getting posted that ask whether ChatGPT generated code was any good, or even valid.

We've seen answers that basically run the provided code or entire question through ChatGPT and post the result without insightful observation. Code dumps have never been valid answers, as per the help on How do I write a good answer?:

Every answer must make at least one insightful observation about the code in the question. Answers that merely provide an alternate solution with no explanation or justification do not constitute valid Code Review answers and may be deleted. In addition to criticisms, pointing out good practices in the code is also a form of helpful feedback.

It doesn't help that, as far as I recall, none of the code dumps in ChatGPT answers had code that actually improved the code (usually it didn't work at all, even if the original did). Some of them even got accepted while being wrong.

A review has always been about the teaching, not the alternative implementation. An alternative implementation that's based on the review can definitely be part of a review - that usually works very well, but the review comes first and must be in the answer. If people want their code to be "optimized" by AI, they can feed it to the AI themselves.