[BUG] Quickstart Ollama External API - GET http://ollama:11434/api/tags "HTTP/1.1 403 Forbidden"

[reysic]

Pre-check

• I have searched the existing issues and none cover this bug.

Description

Following the Quickstart documentation provided here for Ollama External API on macOS results in a 403 error in the PrivateGPT container when attempting to communicate with Ollama.

I've verified that Ollama is running locally by visiting http://localhost:11434/ and receiving the customary "Ollama is running".

private-gpt git:(main) docker compose --profile ollama-api up

WARN[0000] The "HF_TOKEN" variable is not set. Defaulting to a blank string. 
[+] Running 3/0
 ✔ Network private-gpt_default                 Created                                                                                                                                      0.0s 
 ✔ Container private-gpt-ollama-1              Created                                                                                                                                      0.0s 
 ✔ Container private-gpt-private-gpt-ollama-1  Created                                                                                                                                      0.0s 
Attaching to ollama-1, private-gpt-ollama-1
ollama-1              | time="2024-08-22T16:42:04Z" level=info msg="Configuration loaded from flags."
private-gpt-ollama-1  | 16:42:04.266 [INFO    ] private_gpt.settings.settings_loader - Starting application with profiles=['default', 'docker']
private-gpt-ollama-1  | None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
private-gpt-ollama-1  | 16:42:07.647 [INFO    ] private_gpt.components.llm.llm_component - Initializing the LLM in mode=ollama
private-gpt-ollama-1  | 16:42:07.686 [INFO    ]                     httpx - HTTP Request: GET http://ollama:11434/api/tags "HTTP/1.1 403 Forbidden"
private-gpt-ollama-1  | 16:42:07.686 [ERROR   ]  private_gpt.utils.ollama - Failed to connect to Ollama: 
private-gpt-ollama-1  | Traceback (most recent call last):
private-gpt-ollama-1  |   File "/home/worker/app/.venv/lib/python3.11/site-packages/injector/__init__.py", line 798, in get
private-gpt-ollama-1  |     return self._context[key]
private-gpt-ollama-1  |            ~~~~~~~~~~~~~^^^^^
private-gpt-ollama-1  | KeyError: <class 'private_gpt.ui.ui.PrivateGptUi'>
...

Let me know if there's any additional info I can provide that would be helpful, thanks!

Steps to Reproduce

1. git clone https://github.com/zylon-ai/private-gpt.git
2. cd private-gpt
3. OLLAMA_HOST=0.0.0.0 ollama serve
4. docker-compose --profile ollama-api up

Expected Behavior

Successful access to Ollama locally installed on host from PrivateGPT

Actual Behavior

HTTP 403 error following issuance of docker-compose --profile ollama-api up command followed by container exit

Environment

macOS 14.6.1, Ollama 0.3.6, ollama-api profile

Additional Information

No response

Version

0.6.2

Setup Checklist

• Confirm that you have followed the installation instructions in the project’s documentation.
• Check that you are using the latest version of the project.
• Verify disk space availability for model storage and data processing.
• Ensure that you have the necessary permissions to run the project.

NVIDIA GPU Setup Checklist

• Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
• Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
• Ensure proper permissions are set for accessing GPU resources.
• Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)

[mk-relax]

Same here on windows 10. Temporary fix could be to change ollama:11434 with host.docker.internal:11434 in the config files.

[jaluma]

Can you try to disable autopull images in settings.yaml?

[Ycl-Phy]

I am using Ollama 0.3.8 and getting the same issue. I also tried to disable autopull images and no luck.

[MandarUkrulkar]

same issue here

=/tmp/ollama2036586951/runners
ollama_1 | time="2024-09-02T06:18:22Z" level=info msg="Configuration loaded from flags."
private-gpt-ollama_1 | 06:18:23.067 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default', 'docker']
private-gpt-ollama_1 | None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
private-gpt-ollama_1 | 06:18:30.438 [INFO ] private_gpt.components.llm.llm_component - Initializing the LLM in mode=ollama
private-gpt-ollama_1 | 06:18:30.519 [INFO ] httpx - HTTP Request: GET http://ollama:11434/api/tags "HTTP/1.1 503 Service Unavailable"
private-gpt-ollama_1 | 06:18:30.520 [ERROR ] private_gpt.utils.ollama - Failed to connect to Ollama: Service Unavailable