packagesFromDirectoryRecursive

lib.filesystem.packagesFromDirectoryRecursive

Docs pulled from | This Revision | 43 minutes ago

Transform a directory tree containing package files suitable for callPackage into a matching nested attribute set of derivations.

For a directory tree like this:

my-packages
├── a.nix
├── b.nix
├── c
│  ├── my-extra-feature.patch
│  ├── package.nix
│  └── support-definitions.nix
└── my-namespace
   ├── d.nix
   ├── e.nix
   └── f
      └── package.nix

packagesFromDirectoryRecursive will produce an attribute set like this:

# packagesFromDirectoryRecursive {
#   callPackage = pkgs.callPackage;
#   directory = ./my-packages;
# }
{
  a = pkgs.callPackage ./my-packages/a.nix { };
  b = pkgs.callPackage ./my-packages/b.nix { };
  c = pkgs.callPackage ./my-packages/c/package.nix { };
  my-namespace = {
    d = pkgs.callPackage ./my-packages/my-namespace/d.nix { };
    e = pkgs.callPackage ./my-packages/my-namespace/e.nix { };
    f = pkgs.callPackage ./my-packages/my-namespace/f/package.nix { };
  };
}

In particular:

If the input directory contains a package.nix file, then callPackage <directory>/package.nix { } is returned.
Otherwise, the input directory's contents are listed and transformed into an attribute set.
- If a regular file's name has the .nix extension, it is turned into attribute where:
  - The attribute name is the file name without the .nix extension
  - The attribute value is callPackage <file path> { }
- Directories are turned into an attribute where:
  - The attribute name is the name of the directory
  - The attribute value is the result of calling packagesFromDirectoryRecursive { ... } on the directory.
  As a result, directories with no .nix files (including empty directories) will be transformed into empty attribute sets.
- Other files are ignored, including symbolic links to directories and to regular .nix files; this is because nixlang code cannot distinguish the type of a link's target.

Type

packagesFromDirectoryRecursive :: {
  callPackage :: Path -> {} -> a,
  newScope? :: AttrSet -> scope,
  directory :: Path,
} -> AttrSet

Inputs

callPackage: The function used to convert a Nix file's path into a leaf of the attribute set. It is typically the callPackage function, taken from either pkgs or a new scope corresponding to the directory.
newScope: If present, this function is used when recursing into a directory, to generate a new scope. The arguments are updated with the scope's callPackage and newScope functions, so packages can require anything in their scope, or in an ancestor of their scope.
directory: The directory to read package files from.

Examples

Basic use of `lib.packagesFromDirectoryRecursive`

packagesFromDirectoryRecursive {
  inherit (pkgs) callPackage;
  directory = ./my-packages;
}
=> { ... }

In this case, callPackage will only search pkgs for a file's input parameters. In other words, a file cannot refer to another file in the directory in its input parameters.

Create a scope for the nix files found in a directory

packagesFromDirectoryRecursive {
  inherit (pkgs) callPackage newScope;
  directory = ./my-packages;
}
=> { ... }

For example, take the following directory structure:

my-packages
├── a.nix    → { b }: assert b ? b1; ...
└── b
   ├── b1.nix  → { a }: ...
   └── b2.nix

Here, b1.nix can specify { a } as a parameter, which callPackage will resolve as expected. Likewise, a.nix receive an attrset corresponding to the contents of the b directory.

a.nix cannot directly take as inputs packages defined in a child directory, such as b1.

Noogle detected

Aliases

lib.packagesFromDirectoryRecursive

Implementation

The following is the current implementation of this function.

packagesFromDirectoryRecursive =
    let
      inherit (lib)
        concatMapAttrs
        id
        makeScope
        recurseIntoAttrs
        removeSuffix
        ;
      inherit (lib.path) append;

      # Generate an attrset corresponding to a given directory.
      # This function is outside `packagesFromDirectoryRecursive`'s lambda expression,
      #  to prevent accidentally using its parameters.
      processDir =
        { callPackage, directory, ... }@args:
        concatMapAttrs (
          name: type:
          # for each directory entry
          let
            path = append directory name;
          in
          if type == "directory" then
            {
              # recurse into directories
              "${name}" = packagesFromDirectoryRecursive (
                args
                // {
                  directory = path;
                }
              );
            }
          else if type == "regular" && hasSuffix ".nix" name then
            {
              # call .nix files
              "${removeSuffix ".nix" name}" = callPackage path { };
            }
          else if type == "regular" then
            {
              # ignore non-nix files
            }
          else
            throw ''
              lib.filesystem.packagesFromDirectoryRecursive: Unsupported file type ${type} at path ${toString path}
            ''
        ) (builtins.readDir directory);
    in
    {
      callPackage,
      newScope ? throw "lib.packagesFromDirectoryRecursive: newScope wasn't passed in args",
      directory,
    }@args:
    let
      defaultPath = append directory "package.nix";
    in
    if pathExists defaultPath then
      # if `${directory}/package.nix` exists, call it directly
      callPackage defaultPath { }
    else if args ? newScope then
      # Create a new scope and mark it `recurseForDerivations`.
      # This lets the packages refer to each other.
      # See:
      #  [lib.makeScope](https://nixos.org/manual/nixpkgs/unstable/#function-library-lib.customisation.makeScope) and
      #  [lib.recurseIntoAttrs](https://nixos.org/manual/nixpkgs/unstable/#function-library-lib.customisation.makeScope)
      recurseIntoAttrs (
        makeScope newScope (
          self:
          # generate the attrset representing the directory, using the new scope's `callPackage` and `newScope`
          processDir (
            args
            // {
              inherit (self) callPackage newScope;
            }
          )
        )
      )
    else
      processDir args;