packagesFromDirectoryRecursive

lib.packagesFromDirectoryRecursive

Docs pulled from | This Revision | 13 minutes ago

Transform a directory tree containing package files suitable for callPackage into a matching nested attribute set of derivations.

For a directory tree like this:

my-packages
├── a.nix
├── b.nix
├── c
│  ├── my-extra-feature.patch
│  ├── package.nix
│  └── support-definitions.nix
└── my-namespace
   ├── d.nix
   ├── e.nix
   └── f
      └── package.nix

packagesFromDirectoryRecursive will produce an attribute set like this:

# packagesFromDirectoryRecursive {
#   callPackage = pkgs.callPackage;
#   directory = ./my-packages;
# }
{
  a = pkgs.callPackage ./my-packages/a.nix { };
  b = pkgs.callPackage ./my-packages/b.nix { };
  c = pkgs.callPackage ./my-packages/c/package.nix { };
  my-namespace = {
    d = pkgs.callPackage ./my-packages/my-namespace/d.nix { };
    e = pkgs.callPackage ./my-packages/my-namespace/e.nix { };
    f = pkgs.callPackage ./my-packages/my-namespace/f/package.nix { };
  };
}

In particular:

If the input directory contains a package.nix file, then callPackage <directory>/package.nix { } is returned.
Otherwise, the input directory's contents are listed and transformed into an attribute set.
- If a file name has the .nix extension, it is turned into attribute where:
  - The attribute name is the file name without the .nix extension
  - The attribute value is callPackage <file path> { }
- Other files are ignored.
- Directories are turned into an attribute where:
  - The attribute name is the name of the directory
  - The attribute value is the result of calling packagesFromDirectoryRecursive { ... } on the directory.
  As a result, directories with no .nix files (including empty directories) will be transformed into empty attribute sets.

Type

packagesFromDirectoryRecursive :: {
  callPackage :: Path -> {} -> a,
  directory :: Path,
  ...
} -> AttrSet

Inputs

callPackage: The function used to convert a Nix file's path into a leaf of the attribute set. It is typically the callPackage function, taken from either pkgs or a new scope corresponding to the directory.
directory: The directory to read package files from.

Basic use of `lib.packagesFromDirectoryRecursive`

packagesFromDirectoryRecursive {
  inherit (pkgs) callPackage;
  directory = ./my-packages;
}
=> { ... }

In this case, callPackage will only search pkgs for a file's input parameters. In other words, a file cannot refer to another file in the directory in its input parameters.

Create a scope for the nix files found in a directory

lib.makeScope pkgs.newScope (
  self: packagesFromDirectoryRecursive {
    inherit (self) callPackage;
    directory = ./my-packages;
  }
)
=> { ... }

For example, take the following directory structure:

my-packages
├── a.nix    → { b }: assert b ? b1; ...
└── b
   ├── b1.nix  → { a }: ...
   └── b2.nix

Here, b1.nix can specify { a } as a parameter, which callPackage will resolve as expected. Likewise, a.nix receive an attrset corresponding to the contents of the b directory.

a.nix cannot directly take as inputs packages defined in a child directory, such as b1.

As of now, lib.packagesFromDirectoryRecursive cannot create nested scopes for sub-directories.

In particular, files under b/ can only require (as inputs) other files under my-packages, but not to those in the same directory, nor those in a parent directory; e.g, b2.nix cannot directly require b1.

(lib.filesystem.packagesFromDirectoryRecursive)

Noogle detected

Aliases

lib.filesystem.packagesFromDirectoryRecursive

Implementation

The following is the current implementation of this function.

packagesFromDirectoryRecursive =
    {
      callPackage,
      directory,
      ...
    }:
    let
      # Determine if a directory entry from `readDir` indicates a package or
      # directory of packages.
      directoryEntryIsPackage = basename: type:
        type == "directory" || hasSuffix ".nix" basename;

      # List directory entries that indicate packages in the given `path`.
      packageDirectoryEntries = path:
        filterAttrs directoryEntryIsPackage (readDir path);

      # Transform a directory entry (a `basename` and `type` pair) into a
      # package.
      directoryEntryToAttrPair = subdirectory: basename: type:
        let
          path = subdirectory + "/${basename}";
        in
        if type == "regular"
        then
        {
          name = removeSuffix ".nix" basename;
          value = callPackage path { };
        }
        else
        if type == "directory"
        then
        {
          name = basename;
          value = packagesFromDirectory path;
        }
        else
        throw
          ''
            lib.filesystem.packagesFromDirectoryRecursive: Unsupported file type ${type} at path ${toString subdirectory}
          '';

      # Transform a directory into a package (if there's a `package.nix`) or
      # set of packages (otherwise).
      packagesFromDirectory = path:
        let
          defaultPackagePath = path + "/package.nix";
        in
        if pathExists defaultPackagePath
        then callPackage defaultPackagePath { }
        else mapAttrs'
          (directoryEntryToAttrPair path)
          (packageDirectoryEntries path);
    in
    packagesFromDirectory directory;