Deprecate RUST_TEST_* env variables

Like with #139224, this is a documentation-only deprecation for now.
Over time, we can
- warn and then remove on use of unstable environment variables
- warn on use of stable environment variables (no plan to remove due to
  compatibility)

Longer term, we expect test runners, like `cargo test`, to provide the
necessary mechanisms for environmental or persistent configuration (e.g.
using cargo config which supports `.cargo/config.toml` as well as
environment variables).

This would include:
- `RUST_TEST_THREADS`
- `RUST_TEST_NOCAPTURE`
- `RUST_TEST_SHUFFLE` (unstable)
- `RUST_TEST_SHUFFLE_SEED` (unstable)

The primary outcomes for this change are
- Reducing the scope of what is expected for custom test harnesses to
  implement
- Reduce the mechanisms that test runners, like `cargo test`, are
  expected to track when they are being bypassed to protect against
  negative interactions, e.g.  `RUST_TEST_NOCAPTURE=1` when json output
  is being read.

For testing-devex FCP, see rust-lang/testing-devex-team#10

Fixes rust-lang/testing-devex-team#10

History
-------

At each step, I could not find evidence of design discussions on whether
to support CLI, env, or both.  The first env variable seems to come from
the fact that it was being forked out of an existing env variable that
had a much wider scope.

At best, this seems like a way to offer a more persistent configuration
for these flags but environment variables hidden away in libtest is a
bit clunky and this seems like the wrong layer to handle this problem.

**Originally:** `RUST_THREADS` was respected by the Rust runtime and
libextra/test got this for free

**2013:** #7335 suggested splitting `RUST_TEST_TASKS` out
of `RUST_THREADS`.  In that issue and the implementation
(#8823).

**2014:** #13374 ask for support to disable capturing of
stdout/stderr.  `--nocapture` and `RUST_TEST_NOCAPTURE` were added
together.

**2015:** #23525 renamed `RUST_TEST_TASKS` to
`RUST_TEST_THREADS`

**2016:** #25636 asked to configure `RUST_TEST_THREADS`
via `--test-threads` which was implemented in #35414

**2021:** #85440 asked for test randomization which was
implemented in #89082, adding `--shuffle` /
RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED`

Potentially relevant issues
---------------------------

- #74845

Author: epage

library/test/src/cli.rs

@@ -162,18 +162,17 @@ tests whose names contain the filter are run. Multiple filter strings may
 be passed, which will run all tests matching any of the filters.
 
 By default, all tests are run in parallel. This can be altered with the
---test-threads flag or the RUST_TEST_THREADS environment variable when running
-tests (set it to 1).
+--test-threads flag when running tests (set it to 1).
 
-By default, the tests are run in alphabetical order. Use --shuffle or set
-RUST_TEST_SHUFFLE to run the tests in random order. Pass the generated
-"shuffle seed" to --shuffle-seed (or set RUST_TEST_SHUFFLE_SEED) to run the
-tests in the same order again. Note that --shuffle and --shuffle-seed do not
-affect whether the tests are run in parallel.
+By default, the tests are run in alphabetical order. Use --shuffle to run
+the tests in random order. Pass the generated "shuffle seed" to
+--shuffle-seed to run the tests in the same order again. Note that
+--shuffle and --shuffle-seed do not affect whether the tests are run in
+parallel.
 
 All tests have their standard output and standard error captured by default.
-This can be overridden with the --no-capture flag or setting RUST_TEST_NOCAPTURE
-environment variable to a value other than "0". Logging is not captured by default.
+This can be overridden with the --no-capture flag to a value other than "0".
+Logging is not captured by default.
 
 Test Attributes:
 

src/doc/rustc/src/tests/index.md

@@ -164,7 +164,7 @@ Sets the number of threads to use for running tests in parallel. By default,
 uses the amount of concurrency available on the hardware as indicated by
 [`available_parallelism`].
 
-This can also be specified with the `RUST_TEST_THREADS` environment variable.
+Deprecated: this can also be specified with the `RUST_TEST_THREADS` environment variable.
 
 #### `--force-run-in-process`
 
@@ -186,7 +186,7 @@ docs](../../unstable-book/compiler-flags/report-time.html) for more information.
 
 Runs the tests in random order, as opposed to the default alphabetical order.
 
-This may also be specified by setting the `RUST_TEST_SHUFFLE` environment
+Deprecated: this may also be specified by setting the `RUST_TEST_SHUFFLE` environment
 variable to anything but `0`.
 
 The random number generator seed that is output can be passed to
@@ -209,7 +209,7 @@ the tests in the same order both times.
 _SEED_ is any 64-bit unsigned integer, for example, one produced by
 [`--shuffle`](#--shuffle).
 
-This can also be specified with the `RUST_TEST_SHUFFLE_SEED` environment
+Deprecated: this can also be specified with the `RUST_TEST_SHUFFLE_SEED` environment
 variable.
 
 ⚠️ 🚧 This option is [unstable](#unstable-options), and requires the `-Z
@@ -231,7 +231,7 @@ Does not capture the stdout and stderr of the test, and allows tests to print
 to the console. Usually the output is captured, and only displayed if the test
 fails.
 
-This may also be specified by setting the `RUST_TEST_NOCAPTURE` environment
+Deprecated: this may also be specified by setting the `RUST_TEST_NOCAPTURE` environment
 variable to anything but `0`.
 
 `--nocapture` is a deprecated alias for `--no-capture`.