Implement support for fixtures

[roman-kashitsyn]

Use-case

• There is a test suite requires setting up an expensive DB instance.
• All the tests in the suite can use a single DB instance while still being deterministic.

Proposal

In order to support such use-cases, we need to extend the public API a bit. Something like:

struct ParameterizedTestCase<R> {
  name: String,
  options: Options,
  mk_assertion: Box<dyn FnOnce(R) -> GenericAssertion + 'static>,
}

fn test_case_f(
  name: impl ToString,
  f: impl FnOnce(R) -> GenericAssertion + 'static,
) -> ParameterizedTestCase<R>;

fn test_suite_f(
    setup: impl FnOnce() -> Resource,
    teardown: impl FnOnce(Resource),
    test_cases: Vec<ParameterizedTestCase<Resource>>,
) -> TestTree
  where Resource : Clone + Send;

Note: we could allow Resource -> TestTree instead of Resource -> ParameterizedTestCase<Resource>, which would be a nice composable monadic API. However, using a more restrictive signature allows us to:

• Keep test scheduling and execution separate. With a monadic version, we'd have to interleave planning and execution, which makes the code much more complex and error-prone.
• Disallow nested fixtures. Making fixtures independent of each other simplifies the execution logic significantly.

Implications for the execution:

• It becomes impossible to have a single test plan with self-contained tasks, we'll have to intertwine normal task execution, fixtures execution and task construction.
• For every "bundle" with setup/teardown we'll have to track how much tasks are still using the shared resource and release the resource properly once all the tasks using it die. Nesting fixtures are tricky, but seem feasible.

Obvious optimization: do no allocate the resource if the nested test suite is empty/skipped/filtered out.

Open questions:

• Where to execute setup & teardown? If we run just setup/teardown in a separate process, we have to somehow pass the resources to the child tests. If we spawn a sub-process that does setup/teardown and control, reporting results upstream becomes tricky. If we run setup/teardown in the main process, we can stall the event loop for too long.
• Do we want to serialize execution of the parameterized test cases to make them more deterministic?
• What happens with the shared resource if a single test dies? Do we assume that the resource becomes broken?
• How to report failures in fixtures?

[vcm-at-dfinity]

I like it. If I think about it, this is really just shifting a lot of fondue's logic directly into raclette. For the record, this is how we run a fondue "pot", which consists in either an "isolated pot", which can change its enviroment; or a vector of "composable tests", which cannot change its environment and receives a reference to a read-only handle.

To address some of the open questions:

I'd say a ParametrizedTestCase runs in a single child process. It starts by running, setup which produces a R,
then we run each individual test in order providing it with a &R, then we run teardown. If a single test dies, nothing happens to the resource: it only received a reference. In any case, its a failure of the overall test anyway, so we skip running the next tests in line and execute teardown (I wouldn't object if we still run the other test though 🤔 )