12
\$\begingroup\$

What I'd wish for is that I wouldn't have to be a human lint tool.
Pimgd

There are questions on CR where running the code through lint or a similar tool could give useful tips to the poster, especially for beginners who don't know about static analysis tools.

Posting an output from an existing tool wouldn't make for a good answer, but if it was tailored specifically for CR (justifying the review, including helpful links, using friendly language, etc.) maybe it could.

What do you think? Could a bot that posted automated review answers make sense? Or maybe a comment with a link to a report?

\$\endgroup\$
  • \$\begingroup\$ I think we should at least have this discussion, regardless of the outcome. It's important to know what exactly the place of validators and automated reviews is on codereview. \$\endgroup\$ – Mast May 1 '15 at 14:28
  • \$\begingroup\$ Why not post this as a CW answer--not with a bot \$\endgroup\$ – nhgrif May 1 '15 at 14:29
  • 3
    \$\begingroup\$ I for one have run our vba static analysis tool on posted code and used the output as part of my answer. The keyword being "part". I don't think posting such results on their own makes for a very good answer. \$\endgroup\$ – RubberDuck May 1 '15 at 14:30
  • \$\begingroup\$ @RubberDuck Why not? Aren't partial reviews encouraged? \$\endgroup\$ – svick May 1 '15 at 14:31
  • \$\begingroup\$ Yeah. They are, and maybe I'd feel different if it wasn't output from my project. \$\endgroup\$ – RubberDuck May 1 '15 at 14:32
  • 1
    \$\begingroup\$ Would a list of static analysis tools help jumpstart this? \$\endgroup\$ – h.j.k. May 1 '15 at 14:36
14
\$\begingroup\$

I'm not sure that an automated answering bot or whatever would be appropriate, for a couple of reasons. The primary one is this:

It is not a person.

Why does this matter? Well, to me, it matters a lot. Sure, a "lint" type tool can show you errors and potential problems, but it cannot teach. I feel one of the great things in having a real programmer review your code, is they not only can help improve the code, they also help to teach you how to learn better. This, a lint tool cannot do.


There are a few other concerns, mostly logistical in nature:

  • Bot would probably be a PITA to design;

  • Bot would likely need to be a legit SE user (like @Duga) to have privileges to post on the site;

  • Bot would take away opportunities for human users to gain reputation/badges/etc.

  • Bot might confuse new users into thinking it's a real person;

  • How/who manages the bots reputation, badges, etc.?


Instead...

Someone suggested in comments:

Would a list of static analysis tools help jumpstart this? – h.j.k.

I think having a list of static tools in itself would be useful enough for reference.


Update!

Such a list of static tools is being developed here:

What static analysis tools exist that can help pick up points for answering CR questions?

\$\endgroup\$
  • 4
    \$\begingroup\$ "but it cannot teach" Yes. This. ++ \$\endgroup\$ – RubberDuck May 1 '15 at 14:54
  • 1
    \$\begingroup\$ Doesn't teaching just mean explaining why something is wrong? Current lint tools usually don't do that, but a CR-specific one could. \$\endgroup\$ – svick May 1 '15 at 15:00
  • 2
    \$\begingroup\$ A lint tool would not pick up on things like code repetition, design patterns, inefficient (albeit not "wrong") code, how to improve algorithm, etc. etc. \$\endgroup\$ – Phrancis May 1 '15 at 15:12
  • \$\begingroup\$ @sᴉɔuɐɹɥԀ Yes, I'm not expecting automated reviews to be a replacement for human reviews. I expect it to only catch things that can be detected automatically and that are tedious for human reviewers. \$\endgroup\$ – svick May 1 '15 at 16:50
  • \$\begingroup\$ +1 for not human. If static analysis was perfect, there wouldn't be options in them to ignore warnings. \$\endgroup\$ – Comintern May 3 '15 at 14:35
4
\$\begingroup\$

I often use standard tools (e.g. lint, valgrind) or my own tools to help me compose a review. Recently, I suggested creating such tools as a possible Code Challenge.

There's no substitute for an experienced human looking at one's code, but promoting automated tools seems to me to be a good adjunct to Code Review's mission.

I should clarify that I would not be in favor of having bot-posted reviews, but I am in favor of helping programmers help themselves, such as with a list of tools and resources.

One possibility might be to add to the Help Center and describe how to turn on compiler warnings for various different compilers, assemblers, etc. and to give the suggestion that compiling cleanly without errors or warnings before posting code for review is a way to keep the signal-to-noise ratio on Code Review high.

\$\endgroup\$
  • 1
    \$\begingroup\$ Perhaps we should have a community wiki here on meta with a list of such tools? Another option is to add them to the tag wikis of specific languages each tool targets. \$\endgroup\$ – RubberDuck May 1 '15 at 15:20
  • \$\begingroup\$ @RubberDuck I like the tag wiki idea a lot! \$\endgroup\$ – Phrancis May 1 '15 at 15:36
  • \$\begingroup\$ Looks like there's a list of tools growing here on meta as we speak. \$\endgroup\$ – RubberDuck May 1 '15 at 15:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .