Rehash “How secure is my X” cryptography questions

Question

This has been discussed before about 2 years ago: “How secure…” questions - Code Review vs. Information Security SE but it doesn't appear to have led to anything very conclusive.

These questions are pretty common in the cryptography tag and I feel that there are a few fundamental problems with them which I wanted to present to the community for discussion.

1. Is it truly on-topic?

I'm going to quote a recent example:

I just wanted to know if you guys thought if this code was cryptographically secure and also if I could speed it up a little.

Worded like this, it is probably fine. But imagine if you took out "and also if I could speed it up a little", would that then become an issue of "Do I want feedback about any or all facets of the code?"?

For example see this question

[…] since I'm no expert in cryptography (I know the maths behind RSA but thats about it) I have no idea whether this is secure or not. […]

The code works as expected. My questions:

• Is this a correct way to use this module?

• Is this secure?

It is not indicated that the author had the intention to get reviews on anything except the security. Should that (and others like such) have been closed?

2. Is the Help Center too vague on what is a "security issue" in the scope of a code review?

Quoting the Help Center -> On-topic page (emphasis mine):

If you are looking for feedback on a specific working piece of code from your project in the following areas…

• Best practices and design pattern usage
• Security issues
• Performance
• Correctness in unanticipated cases

I think that "Security issues" is too broad of a brush, as it is encapsulating an entire domain of computer science with many researchers and professionals, along with what are some common and relatively simple security holes that can usually be spotted during the course of a code review (SQL injection, XSS attacks, etc.). I'm going to quote an answer by Gareth Rees to the above question:

I'm afraid that your second question ("Is this secure?") is one that you're not likely to get a good answer on here at Code Review. What you seem to be trying to do here is to design a cryptographic protocol, and cryptanalysis requires very specific expertise that I don't think any of the regulars here possess. I certainly don't. Also, to comment on the security of a protocol we have to see the whole protocol.

Sub-questions on point 2:

• Are we setting unrealistic expectations for newcomers that this is the correct site to address any and all security & cryptography issues, as long as it is expressed as reviewable code?
• Should we clarify the Help Center page to limit the scope of what constitues a "Security issue" that can reasonably be addressed in the course of routine reviews? Perhaps provide a link to IT Security's on-topic page?

3. Should we, in certain cases, recommend IT Security Q&A?

Keeping in mind the spirit of "on-topic on other site Y does not make it off-topic on original site X", would in be adviseable in certain cases (in particular where questions fall into what I mentioned in point 1), would it be beneficial both to OPs and to Code Review in general to encourage asking questions there instead of (or in addition to) Code Review, if the question can be reformulated to be on-topic there, i.e., less about the code itself and more about protocols, infrastructutres, tools, etc. relating to information security?

For reference quoting IT Security's on-topic page:

What topics can I ask about here?

IT Security Stack Exchange is for Information Security professionals to discuss protecting assets from threats and vulnerabilities. Topics include, but are not limited to:

• web app hardening
• network security
• social engineering, including phishing
• risk management
• policies
• penetration testing
• security tools
• using cryptography
• incident response
• physically securing the office, datacentre, information assets etc.

Answer 1

With so many Stack Exchange sites, there is bound to be some overlap of scopes. And that's OK. The important principle to note is that a question being a better fit elsewhere is not a sufficient reason for closing it. We normally accept "Is this code secure or not?" questions, and should not carve out an exception for security questions just because Information Security exists, nor should we exclude cryptography questions just because Cryptography SE exists.

Note that cryptanalysis is explicitly off-topic for Cryptography SE — there are limits to how much you can expect the Internet to do for you for free. I don't think that there is a large enough problem with setting expectations that we need to do something to prevent such questions from being asked on Code Review. Such measures to explicitly narrow our scope would likely do more harm than good.

Feel free to advise the poster of a question, though, if you think that it's not likely to receive a useful review. Be sure to explain why it is too complex to review.

Answer 2

I think this can be viewed pretty easily in terms of one basic position:

We review code

If the code itself contains a security vulnerability (e.g., a buffer overflow), finding and commenting on that clearly falls within the scope of reviewing the code.

Likewise, if it's (at least reasonably) clear that the code is intended to implement some particular algorithm (RSA, AES, SHA, etc.) and deviates from the specification of that algorithm, then it's a pretty clear-cut bug, and (again) I think pointing out such a bug is clearly topical.

The more difficult question is when the OP has basically defined what the code should do, but that definition may not provide some quality the OP wanted and hoped it would provide. Here we're starting to get into what looks to me like a fairly grey area. I see two obvious difficulties here. One is a fairly simple matter of logistics: a question usually only provides a fairly minimal description of what the code is really intended to accomplish.

In the case of encryption, a question about "is this algorithm secure" will typically require a fairly detailed threat model before anybody can hope to answer it at all (and full threat-model documentation probably won't fit in a question). "Will it keep my brother from reading my diary" may be quite a bit different from "will it keep the NSA/FBI from reading my data."

At least from my perspective, that leaves three basic possibilities:

1. Your algorithm is clearly broken. I can break it in the following way.
2. Your implementation of well-known algorithm X seems accurate, and I don't know of a significant attack on X.
3. Something in between those.

For better or worse, most are likely to fall into the last category--anything from "I can probably break that, but I'm not interested enough to bother", to "this looks like a worthy modification to algorithm X that prevents published attack Y, and does seem unlikely to open the algorithm to other well known attacks like differential cryptanalysis or linear cryptanalysis."

Ultimately, it comes back to the basic precept though: we review code, not (generally) the basic idea of what the code is intended to do. Yes, we might hit on the latter if there's an obvious issue, but it's more or less incidental, not the real mission of the site.