install --pip-args='--no-dependencies' Error: cannot find package {dep_req.name!r} metadata

[carlosperate]

Describe the bug

Running pipx install --pip-args='--no-dependencies' yotta outputs this error:

Pipx Internal Error: cannot find package {dep_req.name!r} metadata.

I'm trying to use the --no-dependencies pip flag because with this is an old python application and in some environments/python-versions there are issues installing it's unpinned dependencies. So I'd like to first pipx install yotta (a pure python package, with dependencies that are not) and then manually inject pinned versions of its dependencies to the pipx virtualenv.

Basically pipx install yotta will fail in some cases, so I wanted to run pipx install --pip-args='--no-dependencies' yotta + pipx inject yotta -r requirements.txt

How to reproduce

Simplest way might be with a Python docker container:

docker run -it --rm python:3.9-bullseye bash

Inside the container:

pip install pipx

pipx install --pip-args='--no-dependencies' yotta --verbose

Verbose output:

pipx >(setup:801): pipx version is 1.2.1
pipx >(setup:802): Default python interpreter is '/usr/local/bin/python'
pipx >(mkdir:81): creating directory /root/.local/pipx/venvs
pipx >(mkdir:81): creating directory /root/.local/bin
pipx >(mkdir:81): creating directory /root/.local/pipx/.cache
pipx >(package_name_from_spec:322): Determined package name: yotta
pipx >(package_name_from_spec:323): Package name determined in 0.0s
creating virtual environment...
pipx >(run_subprocess:173): running /usr/local/bin/python -m venv --without-pip /root/.local/pipx/venvs/yotta
creating shared libraries...
pipx >(run_subprocess:173): running /usr/local/bin/python -m venv --clear /root/.local/pipx/shared
pipx >(upgrade:91): Upgrading shared libraries in /root/.local/pipx/shared
upgrading shared libraries...
pipx >(run_subprocess:173): running /root/.local/pipx/shared/bin/python -m pip --disable-pip-version-check install --force-reinstall --upgrade pip setuptools wheel
pipx >(run_subprocess:173): running /root/.local/pipx/venvs/yotta/bin/python -c import sysconfig; print(sysconfig.get_path('purelib'))
pipx >(run_subprocess:173): running /root/.local/pipx/shared/bin/python -c import sysconfig; print(sysconfig.get_path('purelib'))
pipx >(run_subprocess:173): running /root/.local/pipx/venvs/yotta/bin/python --version
pipx >(_parsed_package_to_package_or_url:141): cleaned package spec: yotta
installing yotta...
pipx >(run_subprocess:173): running /root/.local/pipx/venvs/yotta/bin/python -m pip install --no-dependencies yotta
pipx >(run_subprocess:173): running <fetch_info_in_venv commands>

pipx >(rmdir:55): removing directory /root/.local/pipx/venvs/yotta
Pipx Internal Error: cannot find package {dep_req.name!r} metadata.

Expected behavior

To install the yotta package into a virtualenv without any of its dependencies.

In the same docker container, running the same commands in a venv works:

python -m venv .venv

source .venv/bin/activate

python -m pip install --no-dependencies yotta

Output

Collecting yotta
  Using cached yotta-0.20.5-py3-none-any.whl
Installing collected packages: yotta
Successfully installed yotta-0.20.5

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip

[carlosperate]

Looking at the error thrown in (which I guess it's meant to be an f-string):

pipx/src/pipx/venv_inspect.py

Lines 130 to 134 in 331a581

	dep_dist = get_dist(dep_req.name, venv_inspect_info.distributions)
	if dep_dist is None:
	raise PipxError(
	"Pipx Internal Error: cannot find package {dep_req.name!r} metadata."
	)

Based on the output of get_dist():

pipx/src/pipx/venv_inspect.py

Lines 36 to 43 in 331a581

	def get_dist(
	package: str, distributions: List[metadata.Distribution]
	) -> Optional[metadata.Distribution]:
	"""Find matching distribution in the canonicalized sense."""
	for dist in distributions:
	if canonicalize_name(dist.metadata["name"]) == canonicalize_name(package):
	return dist
	return None

And updating the function code to print-debug what package names it's comparing:

    print("START")    # print-debug
    for dist in distributions:
        print(canonicalize_name(dist.metadata["name"]), canonicalize_name(package))    # print-debug
        if canonicalize_name(dist.metadata["name"]) == canonicalize_name(package):
            print("FOUND")   # print-debug
            return dist
    print("NOT FOUND")   # print-debug
    return None

It prints the following:

START
yotta yotta
FOUND
START
yotta semantic-version
wheel semantic-version
pip semantic-version
setuptools semantic-version
NOT FOUND

Taking in consideration semantic-version is the first yotta dependency in setup.py, is it possible pipx assumes it always needs to look for the package dependencies? Which in this case won't be present due to the --no-dependencies pip flag.

[dukecat0]

How about running this instead?

pipx install --pip-args='--no-dependencies -r requirements.txt' yotta --verbose

Although the dependencies in requirements.txt won't be stored in the metadata, you can still check them with:

pipx runpip yotta list

pipx inject yotta -r requirements.txt

We are still trying to figure out how should we handle the metadata in #1037.

[carlosperate]

Thanks @dukecat0, yes, that works as a workaround to my issue!

What is pipx policy about potentially incomatible pip flags like this one? If it's hard (or low priority) to fix, maybe a better error message might be useful?

[dukecat0]

Perhaps we can introduce an option like -no-deps?

[carlosperate]

Yeah, that would work 👍
To be fair, this is probably a bit of an edge case, so it'd also be completely understandable if a flag like this is considered out-of-scope for the project, in case useful error messages are a good compromise as well.

[gaborbernat]

Yeah, that would work 👍 To be fair, this is probably a bit of an edge case, so it'd also be completely understandable if a flag like this is considered out-of-scope for the project, in case useful error messages are a good compromise as well.

👍