[BUG] Dependency conflict while using Apple Silicon (Mac M1) Metal GPU.

[harshadvekariya]

Pre-check

• I have searched the existing issues and none cover this bug.

Description

I am trying to use GPU acceleration in Mac M1 with following command.

CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python

It gives library dependency error as follows.

`(.venv) user_mac@user-Mac-mini private-gpt % CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python

Collecting llama-cpp-python
Downloading llama_cpp_python-0.2.90.tar.gz (63.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.8/63.8 MB 6.0 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.5.0 (from llama-cpp-python)
Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting numpy>=1.20.0 (from llama-cpp-python)
Downloading numpy-2.1.1-cp311-cp311-macosx_14_0_arm64.whl.metadata (60 kB)
Collecting diskcache>=5.6.1 (from llama-cpp-python)
Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Collecting jinja2>=2.11.3 (from llama-cpp-python)
Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python)
Downloading MarkupSafe-2.1.5-cp311-cp311-macosx_10_9_universal2.whl.metadata (3.0 kB)
Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)
Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
Downloading numpy-2.1.1-cp311-cp311-macosx_14_0_arm64.whl (5.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.4/5.4 MB 7.1 MB/s eta 0:00:00
Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Downloading MarkupSafe-2.1.5-cp311-cp311-macosx_10_9_universal2.whl (18 kB)
Building wheels for collected packages: llama-cpp-python
Building wheel for llama-cpp-python (pyproject.toml) ... done
Created wheel for llama-cpp-python: filename=llama_cpp_python-0.2.90-cp311-cp311-macosx_14_0_arm64.whl size=2990281 sha256=670e7a3b3fe3224461cd874ae35fff389404d926c4241df093274156c92cb52f
Stored in directory: /private/var/folders/jl/ypzt6tgx42l55x1rfp03h19m0000gn/T/pip-ephem-wheel-cache-e7jvm2kd/wheels/f8/bd/3d/dcb25387572a6fe6c9c2953597c4c4d6dd985a425475da444d
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, numpy, MarkupSafe, diskcache, jinja2, llama-cpp-python
Attempting uninstall: typing-extensions
Found existing installation: typing_extensions 4.12.2
Uninstalling typing_extensions-4.12.2:
Successfully uninstalled typing_extensions-4.12.2
Attempting uninstall: numpy
Found existing installation: numpy 2.1.1
Uninstalling numpy-2.1.1:
Successfully uninstalled numpy-2.1.1
Attempting uninstall: MarkupSafe
Found existing installation: MarkupSafe 2.1.5
Uninstalling MarkupSafe-2.1.5:
Successfully uninstalled MarkupSafe-2.1.5
Attempting uninstall: diskcache
Found existing installation: diskcache 5.6.3
Uninstalling diskcache-5.6.3:
Successfully uninstalled diskcache-5.6.3
Attempting uninstall: jinja2
Found existing installation: Jinja2 3.1.4
Uninstalling Jinja2-3.1.4:
Successfully uninstalled Jinja2-3.1.4
Attempting uninstall: llama-cpp-python
Found existing installation: llama_cpp_python 0.2.90
Uninstalling llama_cpp_python-0.2.90:
Successfully uninstalled llama_cpp_python-0.2.90
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
llama-index-core 0.10.58 requires numpy<2.0.0, but you have numpy 2.1.1 which is incompatible.
Successfully installed MarkupSafe-2.1.5 diskcache-5.6.3 jinja2-3.1.4 llama-cpp-python-0.2.90 numpy-2.1.1 typing-extensions-4.12.2

Steps to Reproduce

Install make and python 3.11 if not installed

brew install make
pyenv install 3.11

Install Ollama

https://ollama.com/

Clone

git clone https://github.com/zylon-ai/private-gpt

Open another terminal and run

ollama serve

Open Visual studio, open project, make virtual env, activate with

python3.11 -m venv .venv

Install Private GPT with Ollama

poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"

Enable Metal GPU (Here comes the Error!)

CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python

Note

The server works fine without using gpu with following command. I have also tried to reproduce it with local and ollama, both gives same error. I have tried downgrading numpy, but above line reinstalls it to newest version and error persists.
PGPT_PROFILES=ollama make run

Expected Behavior

Metal GPU enabled on Apple silicon

Actual Behavior

Error enabling the metal gpu

Environment

OSX, Mac M1 processor, Python 3.11.9

Additional Information

No response

Version

0.6.2

Setup Checklist

• Confirm that you have followed the installation instructions in the project’s documentation.
• Check that you are using the latest version of the project.
• Verify disk space availability for model storage and data processing.
• Ensure that you have the necessary permissions to run the project.

NVIDIA GPU Setup Checklist

• Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
• Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
• Ensure proper permissions are set for accessing GPU resources.
• Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)

[jaluma]

Try this:
CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python numpy==1.26.0