Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

worker_process_entrypoint FAILED #10

Open
ahsaan-habib opened this issue Jul 24, 2024 · 1 comment
Open

worker_process_entrypoint FAILED #10

ahsaan-habib opened this issue Jul 24, 2024 · 1 comment

Comments

@ahsaan-habib
Copy link
Contributor

ahsaan-habib commented Jul 24, 2024

I have tried with my ubuntu 22.04 OS but it gives following error.

E0724 19:33:34.565000 128818126430656 torch/distributed/elastic/multiprocessing/api.py:702] failed (exitcode: -9) local_rank: 0 (pid: 30194) of fn: worker_process_entrypoint (start_method: fork)
E0724 19:33:34.565000 128818126430656 torch/distributed/elastic/multiprocessing/api.py:702] Traceback (most recent call last):
E0724 19:33:34.565000 128818126430656 torch/distributed/elastic/multiprocessing/api.py:702] File "/home/aleya/Work/Habibi/llama/venv/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 659, in _poll
E0724 19:33:34.565000 128818126430656 torch/distributed/elastic/multiprocessing/api.py:702] self._pc.join(-1)
E0724 19:33:34.565000 128818126430656 torch/distributed/elastic/multiprocessing/api.py:702] File "/home/aleya/Work/Habibi/llama/venv/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 170, in join
E0724 19:33:34.565000 128818126430656 torch/distributed/elastic/multiprocessing/api.py:702] raise ProcessExitedException(
E0724 19:33:34.565000 128818126430656 torch/distributed/elastic/multiprocessing/api.py:702] torch.multiprocessing.spawn.ProcessExitedException: process 0 terminated with signal SIGKILL
Process ForkProcess-1:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/Habib/llama/venv/lib/python3.10/site-packages/llama_toolchain/inference/parallel_utils.py", line 175, in launch_dist_group
elastic_launch(launch_config, entrypoint=worker_process_entrypoint)(
File "/home/Habib/llama/venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 133, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/Habib/llama/venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

======================================================
worker_process_entrypoint FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2024-07-24_19:33:34
host : Habib-Stealth-15M-A11SDK
rank : 0 (local_rank: 0)
exitcode : -9 (pid: 30194)
error_file: <N/A>
traceback : Signal 9 (SIGKILL) received by PID 30194

@AleyaSiddika
Copy link

same problem in my ubuntu OS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants