throwTestFailures

lib.debug.throwTestFailures

Docs pulled from | This Revision | 9 minutes ago

Pretty-print a list of test failures.

This takes an attribute set containing failures (a list of test failures produced by runTests) and pretty-prints each failing test, before throwing an error containing the raw test data as JSON.

If the input list is empty, null is returned.

Inputs

failures: A list of test failures (produced runTests), each containing name, expected, and result attributes.

Type

throwTestFailures :: {
  failures = [
    {
      name :: String;
      expected :: a;
      result :: a;
    }
  ];
}
->
null

Examples

`lib.debug.throwTestFailures` usage example

throwTestFailures {
  failures = [
    {
      name = "testDerivation";
      expected = derivation {
        name = "a";
        builder = "bash";
        system = "x86_64-linux";
      };
      result = derivation {
        name = "b";
        builder = "bash";
        system = "x86_64-linux";
      };
    }
  ];
}
->
trace: FAIL testDerivation:
  Expected: <derivation a>
    Result: <derivation b>

error:
       … while evaluating the file '...':

       … caused by explicit throw
         at /nix/store/.../lib/debug.nix:528:7:
          527|       in
          528|       throw (
             |       ^
          529|         builtins.seq traceFailures (

       error: 1 tests failed:
       - testDerivation

       [{"expected":"/nix/store/xh7kyqp69mxkwspmi81a94m9xx74r8dr-a","name":"testDerivation","result":"/nix/store/503l84nir4zw57d1shfhai25bxxn16c6-b"}]
null

Noogle detected

Implementation

The following is the current implementation of this function.

throwTestFailures =
    {
      failures,
      description ? "tests",
      ...
    }:
    if failures == [ ] then
      null
    else
      let
        toPretty =
          value:
          # Thanks to @Ma27 for this:
          #
          # > The `unsafeDiscardStringContext` is useful when the `toPretty`
          # > stumbles upon a derivation that would be realized without it (I
          # > ran into the problem when writing a test for a flake helper where
          # > I creating a bunch of "mock" derivations for different systems
          # > and Nix then tried to build those when the error-string got
          # > forced).
          #
          # See: https://github.com/NixOS/nixpkgs/pull/416207#discussion_r2145942389
          builtins.unsafeDiscardStringContext (generators.toPretty { allowPrettyValues = true; } value);

        failureToPretty = failure: ''
          FAIL ${toPretty failure.name}:
          Expected:
          ${toPretty failure.expected}

          Result:
          ${toPretty failure.result}
        '';

        traceFailures = foldl' (_accumulator: failure: traceVal (failureToPretty failure)) null failures;
      in
      throw (
        builtins.seq traceFailures (
          "${builtins.toString (builtins.length failures)} ${description} failed:\n- "
          + (concatMapStringsSep "\n- " (failure: failure.name) failures)
          + "\n\n"
          + builtins.toJSON failures
        )
      );