Tidy rule against `issue-*` filenames in tests?

[workingjubilee]

These filenames are really quite confounding as they mean that everything requires looking at its contents to understand what it is about. This kind of filename is only slightly better than naming it test.rs. We may want to lint against adding new ones.

[Mark-Simulacrum]

For random ICE tests and such it's not always clear we have a better name - I think before we lint we should have guidelines on good alternatives. (Or decide that if you can't name it you shouldn't add it, but that seems strictly worse).

[workingjubilee]

Yes, we have to at least have a minimum bar of "what else, then?" before we discourage them programmatically. Though perhaps we could also say that, for instance, tests in some specific folder, like rust/tests/ui/ice/**/*, are excepted from such a lint.

[jieyouxu]

I think sometimes having the issue number is useful for context, but

• issue-xxx.rs where it's only the issue number and no other info is awful for searching/discoverability and almost certainly deserve a tidy error.
• issue-xxx-what-it-tests.rs seem alright?

[workingjubilee]

Test files named issue-123456-ice.rs would mean the last 3 letters of the (pre-extension) filename give us more data about the test's actual purpose than the other 13 (which would serve as indirect pointer only), but perhaps that would be sufficient.

[Noratrieb]

I think every ICE has a better potential name than issue-XXX. At least for the ones I've fixed. I think including the issue number is useful, but I prefer having it as a comment in the file instead of the filename.

[asquared31415]

@rustbot claim

[asquared31415]

Initial investigation:
3063/15424 (~19.86%) *.rs ui test files match ^issue[-_ ]?\d+$
another 1349 files (total ~28.60%) contain that pattern in addition to some other text, where they should probably omit it in favor of a comment.

Wow that's a lot of tests. I'm not sure it's feasible for this to retroactively apply to existing tests because each of these tests requires manual investigation to determine what it's meant to be testing. I suspect that a large number of the tests are very old, and the older tests, especially pre-1.0, often need much deeper investigation. This at least cannot be in one large PR, and we'd certainly want to stop new tests from being like this as soon as possible.

[workingjubilee]

I believe that we should grandfather clause all the existing ones rather than simply forcing them all to migrate immediately.