Sniff to check that test class names and test file names comply with PHPUnit naming conventions

[jrfnl]

Is your feature request related to a problem?

To quote myself in Trac ticket https://core.trac.wordpress.org/ticket/62004:

As things are, none of the WordPress test classes comply with the naming conventions expected by PHPUnit and while they will still run on PHPUnit 9 (and 10 for that matter), this is no longer the case on PHPUnit 11 where any and all tests which do not comply with the naming conventions will no longer run.

For automatic recognition of the tests, test classes have to follow the following naming conventions:

1. Class name [SomethingUnderTest]Test - take note of the Test suffix.
2. The file name for the test class MUST match the class name EXACTLY in a case-sensitive manner.
   In other words, test file names have to comply largely with PSR4 file name regulations, though underscores in the names are still allowed.

With some config tweaks, the first rule could be bypassed, however, the second rule cannot be bypassed.

As all test classes will need to be touched/changed/split to address problem 4 anyway, I would strongly recommend for the new test classes to be made fully compatible with the PHPUnit naming conventions.
This will prevent having to re-do this exercise yet again in the future if the bypass for the first rule would be made impossible.

Describe the solution you'd like

In other words, while it would not be possible at this point in time to introduce a sniff to check that test classes and test files comply with the PHPUnit naming conventions (as it would fail the CS build of WP Core hard), this will need to be addressed in the next year or so.

A sniff for this needs to be available by the time WordPress starts supporting PHPUnit 11 to prevent new issues being (re-)introduced into the codebase.

I still have a plan to create a PHPUnit related external PHPCS standard. That standard would be the best place for a sniff like this.

If that standard would be made available/public in time, we can add it as a new dependency and use a sniff from that standard (ideally).
If the standard is not (yet) available, we will need to think of a WPCS native solution for the time being.

As for what should and shouldn't be flagged, let's quote myself again:

What this means in practice (example):

In the current situation, the test class Tests_Compat_ArrayKeyLast is in a file called tests/compat/arrayKeyLast.php.
This class will need to be renamed to Compat_ArrayKeyLast_Test and the file will need to be renamed to tests/compat/Compat_ArrayKeyLast_Test.php.

Or, even better, we could choose to use namespaces in the tests (also see ticket #53010) and to rename the class to WordPress\Tests\Compat\ArrayKeyLastTest and the file to tests/compat/ArrayKeyLastTest.php.

Additional context (optional)

Related to:

• FileNameSniff: do not apply to test files #1995
• https://core.trac.wordpress.org/ticket/53010
• https://core.trac.wordpress.org/ticket/62004

• I intend to create a pull request to implement this feature.