Rate Limit #434

Closed Answered by jamesbraza

derspotter asked this question in Q&A

derspotter
Sep 18, 2024

Hi, thank you for the great project.
I am running into rate limiting issues and dont really get why. This is my output:
`pqa ask 'How does ai regulation in Chaina differ from ai regulation in the eu?'
[13:52:43] Starting paper search for 'AI regulation in China'.
Starting paper search for 'AI regulation in the EU'.
New file to index: w32430.pdf...
New file to index: Bill of Rights AI.pdf...
[13:52:44] New file to index: 815d75d6-3fbc-4877-9540-fac65e69b171-meca.pdf...
New file to index: ssrn_id4414065_code2763040.pdf...
[13:52:45] New file to index: WP2023-064-BUGHIN-how.pdf...
New file to index: Bill of Rights AI.pdf...
New file to index: 815d75d6-3fbc-4877-9540-fac65e69b171-meca.pdf...
New file to index: ssrn_id4756641_code258113.pdf...
[13:52:46] New file to index: WP2023-064-BUGHIN-how.pdf...
New file to index: w31161.pdf...
New file to index: 42-06.pdf...
New file to index: ssrn_id4756641_code258113.pdf...
New file to index:
I_think_this_is_the_most_disruptive_technology_Exp.pdf...
[13:52:47] New file to index: ssrn_id4814872_code2226140.pdf...
New file to index: ssrn-4754950.pdf...
New file to index: WEF_Jobs_of_Tomorrow_Generative_AI_2023.pdf...
New file to index: ssrn_id4814872_code2226140.pdf...
New file to index: ssrn_id4802313_code332758.pdf...
[13:52:48] New file to index: w31161.pdf...
New file to index: ssrn_id4802313_code332758.pdf...
New file to index: ssrn-4754950.pdf...
New file to index: w32430.pdf...
New file to index: 42-06.pdf...
[13:52:49] New file to index: WEF_Jobs_of_Tomorrow_Generative_AI_2023.pdf...
New file to index:
I_think_this_is_the_most_disruptive_technology_Exp.pdf...
New file to index: ssrn_id4414065_code2763040.pdf...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.

Exception Group Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
| content = await tool._tool_fn(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/tools.py", line 127, in paper_search
| index = await get_directory_index(settings=self.settings)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 481, in get_directory_index
| async with anyio.create_task_group() as tg:
| File "/home/jay/paperqa/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 961, in acompletion
| headers, response = await self.make_openai_chat_completion_request(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 658, in make_openai_chat_completion_request
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 646, in make_openai_chat_completion_request
| await openai_aclient.chat.completions.with_raw_response.create(
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_legacy_response.py", line 370, in wrapped
| return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1412, in create
| return await self._post(
| ^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
| return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
| return await self._request(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
| raise self._make_status_error_from_response(err.response) from None
| openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 28899, Requested 1818. Please try again in 1.434s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
| response = await init_response
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 1008, in acompletion
| raise OpenAIError(
| litellm.llms.OpenAI.openai.OpenAIError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 28899, Requested 1818. Please try again in 1.434s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 377, in process_file
| await tmp_docs.aadd(
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/docs.py", line 273, in aadd
| result = await llm_model.run_prompt(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 182, in run_prompt
| return await self._run_chat(
| ^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 235, in _run_chat
| chunk = await self.achat(messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 452, in achat
| response = await self.router.acompletion(self.name, messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
| response = await self.async_function_with_fallbacks(**kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
| response = await self.async_function_with_retries(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
| response = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
| response = await _response
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
| result = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
| raise exception_type(
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 6556, in exception_type
| raise RateLimitError(
| litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 28899, Requested 1818. Please try again in 1.434s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
| Received Model Group=gpt-4o-2024-08-06
| Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
+------------------------------------

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

/usr/lib/python3.11/asyncio/base_events.py:1907: RuntimeWarning: coroutine 'OpenAIChatCompletion.acompletion' was never awaited
handle = self._ready.popleft()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Failed to execute tool call for tool paper_search.

Exception Group Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
| content = await tool._tool_fn(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/tools.py", line 127, in paper_search
| index = await get_directory_index(settings=self.settings)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 481, in get_directory_index
| async with anyio.create_task_group() as tg:
| File "/home/jay/paperqa/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 961, in acompletion
| headers, response = await self.make_openai_chat_completion_request(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 658, in make_openai_chat_completion_request
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 646, in make_openai_chat_completion_request
| await openai_aclient.chat.completions.with_raw_response.create(
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_legacy_response.py", line 370, in wrapped
| return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1412, in create
| return await self._post(
| ^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
| return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
| return await self._request(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
| raise self._make_status_error_from_response(err.response) from None
| openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 28857, Requested 1818. Please try again in 1.35s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
| response = await init_response
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 1008, in acompletion
| raise OpenAIError(
| litellm.llms.OpenAI.openai.OpenAIError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 28857, Requested 1818. Please try again in 1.35s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 377, in process_file
| await tmp_docs.aadd(
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/docs.py", line 273, in aadd
| result = await llm_model.run_prompt(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 182, in run_prompt
| return await self._run_chat(
| ^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 235, in _run_chat
| chunk = await self.achat(messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 452, in achat
| response = await self.router.acompletion(self.name, messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
| response = await self.async_function_with_fallbacks(**kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
| response = await self.async_function_with_retries(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
| response = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
| response = await _response
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
| result = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
| raise exception_type(
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 6556, in exception_type
| raise RateLimitError(
| litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 28857, Requested 1818. Please try again in 1.35s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
| Received Model Group=gpt-4o-2024-08-06
| Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
+------------------------------------
[13:52:52] Starting paper search for 'AI regulation in China'.
Starting paper search for 'AI regulation in the EU'.
New file to index:
I_think_this_is_the_most_disruptive_technology_Exp.pdf...
New file to index: 815d75d6-3fbc-4877-9540-fac65e69b171-meca.pdf...
New file to index: WEF_Jobs_of_Tomorrow_Generative_AI_2023.pdf...
[13:52:53] New file to index: ssrn_id4814872_code2226140.pdf...
New file to index: 42-06.pdf...
New file to index: ssrn_id4756641_code258113.pdf...
New file to index: ssrn_id4414065_code2763040.pdf...
[13:52:54] New file to index: Bill of Rights AI.pdf...
New file to index: WP2023-064-BUGHIN-how.pdf...
New file to index: ssrn_id4802313_code332758.pdf...
New file to index: ssrn-4754950.pdf...
[13:52:55] New file to index: w31161.pdf...
New file to index: w32430.pdf...
New file to index: ssrn_id4756641_code258113.pdf...
New file to index: WP2023-064-BUGHIN-how.pdf...
[13:52:56] New file to index: w32430.pdf...
New file to index: ssrn_id4802313_code332758.pdf...
New file to index: 42-06.pdf...
New file to index: 815d75d6-3fbc-4877-9540-fac65e69b171-meca.pdf...
New file to index: I_think_this_is_the_most_disruptive_technology_Exp.pdf...
New file to index: ssrn-4754950.pdf...
[13:52:57] New file to index: Bill of Rights AI.pdf...
New file to index: w31161.pdf...
New file to index: ssrn_id4814872_code2226140.pdf...
[13:52:58] New file to index: ssrn_id4414065_code2763040.pdf...
New file to index: WEF_Jobs_of_Tomorrow_Generative_AI_2023.pdf...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.

Exception Group Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
| content = await tool._tool_fn(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/tools.py", line 127, in paper_search
| index = await get_directory_index(settings=self.settings)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 481, in get_directory_index
| async with anyio.create_task_group() as tg:
| File "/home/jay/paperqa/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 961, in acompletion
| headers, response = await self.make_openai_chat_completion_request(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 658, in make_openai_chat_completion_request
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 646, in make_openai_chat_completion_request
| await openai_aclient.chat.completions.with_raw_response.create(
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_legacy_response.py", line 370, in wrapped
| return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1412, in create
| return await self._post(
| ^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
| return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
| return await self._request(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
| raise self._make_status_error_from_response(err.response) from None
| openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29509, Requested 1815. Please try again in 2.648s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
| response = await init_response
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 1008, in acompletion
| raise OpenAIError(
| litellm.llms.OpenAI.openai.OpenAIError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29509, Requested 1815. Please try again in 2.648s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 377, in process_file
| await tmp_docs.aadd(
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/docs.py", line 273, in aadd
| result = await llm_model.run_prompt(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 182, in run_prompt
| return await self._run_chat(
| ^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 235, in _run_chat
| chunk = await self.achat(messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 452, in achat
| response = await self.router.acompletion(self.name, messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
| response = await self.async_function_with_fallbacks(**kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
| response = await self.async_function_with_retries(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
| response = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
| response = await _response
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
| result = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
| raise exception_type(
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 6556, in exception_type
| raise RateLimitError(
| litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29509, Requested 1815. Please try again in 2.648s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
| Received Model Group=gpt-4o-2024-08-06
| Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
+------------------------------------

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.

Exception Group Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
| content = await tool._tool_fn(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/tools.py", line 127, in paper_search
| index = await get_directory_index(settings=self.settings)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 481, in get_directory_index
| async with anyio.create_task_group() as tg:
| File "/home/jay/paperqa/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (2 sub-exceptions)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 961, in acompletion
| headers, response = await self.make_openai_chat_completion_request(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 658, in make_openai_chat_completion_request
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 646, in make_openai_chat_completion_request
| await openai_aclient.chat.completions.with_raw_response.create(
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_legacy_response.py", line 370, in wrapped
| return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1412, in create
| return await self._post(
| ^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
| return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
| return await self._request(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
| raise self._make_status_error_from_response(err.response) from None
| openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29469, Requested 1812. Please try again in 2.562s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
| response = await init_response
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 1008, in acompletion
| raise OpenAIError(
| litellm.llms.OpenAI.openai.OpenAIError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29469, Requested 1812. Please try again in 2.562s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 377, in process_file
| await tmp_docs.aadd(
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/docs.py", line 273, in aadd
| result = await llm_model.run_prompt(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 182, in run_prompt
| return await self._run_chat(
| ^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 235, in _run_chat
| chunk = await self.achat(messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 452, in achat
| response = await self.router.acompletion(self.name, messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
| response = await self.async_function_with_fallbacks(**kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
| response = await self.async_function_with_retries(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
| response = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
| response = await _response
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
| result = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
| raise exception_type(
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 6556, in exception_type
| raise RateLimitError(
| litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29469, Requested 1812. Please try again in 2.562s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
| Received Model Group=gpt-4o-2024-08-06
| Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
+---------------- 2 ----------------
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 961, in acompletion
| headers, response = await self.make_openai_chat_completion_request(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 658, in make_openai_chat_completion_request
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 646, in make_openai_chat_completion_request
| await openai_aclient.chat.completions.with_raw_response.create(
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_legacy_response.py", line 370, in wrapped
| return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1412, in create
| return await self._post(
| ^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
| return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
| return await self._request(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
| raise self._make_status_error_from_response(err.response) from None
| openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29459, Requested 1812. Please try again in 2.542s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
| response = await init_response
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 1008, in acompletion
| raise OpenAIError(
| litellm.llms.OpenAI.openai.OpenAIError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29459, Requested 1812. Please try again in 2.542s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 377, in process_file
| await tmp_docs.aadd(
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/docs.py", line 273, in aadd
| result = await llm_model.run_prompt(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 182, in run_prompt
| return await self._run_chat(
| ^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 235, in _run_chat
| chunk = await self.achat(messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 452, in achat
| response = await self.router.acompletion(self.name, messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
| response = await self.async_function_with_fallbacks(**kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
| response = await self.async_function_with_retries(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
| response = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
| response = await _response
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
| result = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
| raise exception_type(
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 6556, in exception_type
| raise RateLimitError(
| litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29459, Requested 1812. Please try again in 2.542s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
| Received Model Group=gpt-4o-2024-08-06
| Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
+------------------------------------
[13:53:01] Starting paper search for 'AI regulation in China'.
Starting paper search for 'AI regulation in the EU'.
New file to index: ssrn_id4814872_code2226140.pdf...
New file to index: 815d75d6-3fbc-4877-9540-fac65e69b171-meca.pdf...
New file to index: WEF_Jobs_of_Tomorrow_Generative_AI_2023.pdf...
[13:53:02] New file to index: WP2023-064-BUGHIN-how.pdf...
New file to index: ssrn_id4414065_code2763040.pdf...
New file to index: 42-06.pdf...
New file to index: ssrn_id4756641_code258113.pdf...
[13:53:03] New file to index: I_think_this_is_the_most_disruptive_technology_Exp.pdf...
New file to index: ssrn_id4802313_code332758.pdf...
New file to index: ssrn_id4802313_code332758.pdf...
New file to index: ssrn-4754950.pdf...
New file to index: ssrn_id4814872_code2226140.pdf...
New file to index: 42-06.pdf...
[13:53:04] New file to index: ssrn-4754950.pdf...
New file to index: Bill of Rights AI.pdf...
New file to index: Bill of Rights AI.pdf...
[13:53:05] New file to index: I_think_this_is_the_most_disruptive_technology_Exp.pdf...
New file to index: w31161.pdf...
New file to index: ssrn_id4414065_code2763040.pdf...
[13:53:06] New file to index: WP2023-064-BUGHIN-how.pdf...
New file to index: WEF_Jobs_of_Tomorrow_Generative_AI_2023.pdf...
New file to index: 815d75d6-3fbc-4877-9540-fac65e69b171-meca.pdf...
New file to index: w32430.pdf...
New file to index: ssrn_id4756641_code258113.pdf...
[13:53:07] New file to index: w32430.pdf...
New file to index: w31161.pdf...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.

Exception Group Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
| content = await tool._tool_fn(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/tools.py", line 127, in paper_search
| index = await get_directory_index(settings=self.settings)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 481, in get_directory_index
| async with anyio.create_task_group() as tg:
| File "/home/jay/paperqa/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 961, in acompletion
| headers, response = await self.make_openai_chat_completion_request(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 658, in make_openai_chat_completion_request
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 646, in make_openai_chat_completion_request
| await openai_aclient.chat.completions.with_raw_response.create(
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_legacy_response.py", line 370, in wrapped
| return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1412, in create
| return await self._post(
| ^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
| return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
| return await self._request(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
| raise self._make_status_error_from_response(err.response) from None
| openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29576, Requested 1827. Please try again in 2.806s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
| response = await init_response
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 1008, in acompletion
| raise OpenAIError(
| litellm.llms.OpenAI.openai.OpenAIError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29576, Requested 1827. Please try again in 2.806s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 377, in process_file
| await tmp_docs.aadd(
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/docs.py", line 273, in aadd
| result = await llm_model.run_prompt(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 182, in run_prompt
| return await self._run_chat(
| ^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 235, in _run_chat
| chunk = await self.achat(messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 452, in achat
| response = await self.router.acompletion(self.name, messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
| response = await self.async_function_with_fallbacks(**kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
| response = await self.async_function_with_retries(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
| response = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
| response = await _response
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
| result = await original_function(*args, **kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
| raise exception_type(
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 6556, in exception_type
| raise RateLimitError(
| litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29576, Requested 1827. Please try again in 2.806s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
| Received Model Group=gpt-4o-2024-08-06
| Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
+------------------------------------

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.

Exception Group Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
| content = await tool._tool_fn(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/tools.py", line 127, in paper_search
| index = await get_directory_index(settings=self.settings)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 481, in get_directory_index
| async with anyio.create_task_group() as tg:
| File "/home/jay/paperqa/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 961, in acompletion
| headers, response = await self.make_openai_chat_completion_request(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 658, in make_openai_chat_completion_request
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 646, in make_openai_chat_completion_request
| await openai_aclient.chat.completions.with_raw_response.create(
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_legacy_response.py", line 370, in wrapped
| return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1412, in create
| return await self._post(
| ^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
| return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
| return await self._request(
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
| raise self._make_status_error_from_response(err.response) from None
| openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29469, Requested 1817. Please try again in 2.572s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
| response = await init_response
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/llms/OpenAI/openai.py", line 1008, in acompletion
| raise OpenAIError(
| litellm.llms.OpenAI.openai.OpenAIError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29469, Requested 1817. Please try again in 2.572s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/search.py", line 377, in process_file
| await tmp_docs.aadd(
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/docs.py", line 273, in aadd
| result = await llm_model.run_prompt(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 182, in run_prompt
| return await self._run_chat(
| ^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 235, in _run_chat
| chunk = await self.achat(messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/paperqa/llms.py", line 452, in achat
| response = await self.router.acompletion(self.name, messages)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
| response = await self.async_function_with_fallbacks(kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
| response = await self.async_function_with_retries(*args, kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
| raise original_exception
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
| response = await original_function(*args, kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
| response = await _response
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
| result = await original_function(*args, kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
| raise exception_type(
| ^^^^^^^^^^^^^^^
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
| raise e
| File "/home/jay/paperqa/lib/python3.11/site-packages/litellm/utils.py", line 6556, in exception_type
| raise RateLimitError(
| litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-0pvt6PglePNp3ntgHsbE6lvY on tokens per min (TPM): Limit 30000, Used 29469, Requested 1817. Please try again in 2.572s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
| Received Model Group=gpt-4o-2024-08-06
| Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
+------------------------------------
[13:53:10] Retrying in 0.0 seconds as it raised MalformedMessageError: Unexpected shape of LiteLLM model response ModelResponse(id='chatcmpl-A8nX34GrldBDUZQJhB1PHnHSxqUNs',
choices=[Choices(finish_reason='stop', index=0, message=Message(content='It seems there is an issue with accessing the paper search tool at the moment. Unfortunately, I am unable to retrieve the
necessary information to answer the question about AI regulation differences between China and the EU. You might want to try again later or consult other resources for this information.',
role='assistant', tool_calls=None, function_call=None))], created=1726660389, model='gpt-4o-2024-08-06', object='chat.completion', system_fingerprint='fp_9e15ccd6a4', usage=Usage(completion_tokens=56,
prompt_tokens=873, total_tokens=929, completion_tokens_details=CompletionTokensDetails(reasoning_tokens=0)), service_tier=None)..
[13:53:11] Retrying in 0.0 seconds as it raised MalformedMessageError: Unexpected shape of LiteLLM model response ModelResponse(id='chatcmpl-A8nX4OzYL6MnBVYvfAXYMuuOHbya7',
choices=[Choices(finish_reason='stop', index=0, message=Message(content='It seems there is an issue with retrieving papers at the moment. Unfortunately, I am unable to gather the necessary evidence to
answer the question about how AI regulation in China differs from AI regulation in the EU. You might want to try again later or consult other resources for this information.', role='assistant',
tool_calls=None, function_call=None))], created=1726660390, model='gpt-4o-2024-08-06', object='chat.completion', system_fingerprint='fp_9e15ccd6a4', usage=Usage(completion_tokens=57,
prompt_tokens=873, total_tokens=930, completion_tokens_details=CompletionTokensDetails(reasoning_tokens=0)), service_tier=None)..
[13:53:12] Retrying in 0.0 seconds as it raised MalformedMessageError: Unexpected shape of LiteLLM model response ModelResponse(id='chatcmpl-A8nX5znQsnkcZkfs6HQ4yIO4mZFMn',
choices=[Choices(finish_reason='stop', index=0, message=Message(content='It seems there is an issue with retrieving papers at the moment. Unfortunately, I am unable to gather the necessary evidence to
answer the question about AI regulation differences between China and the EU. You might want to try again later or consult other resources for this information.', role='assistant', tool_calls=None,
function_call=None))], created=1726660391, model='gpt-4o-2024-08-06', object='chat.completion', system_fingerprint='fp_9e15ccd6a4', usage=Usage(completion_tokens=53, prompt_tokens=873,
total_tokens=926, completion_tokens_details=CompletionTokensDetails(reasoning_tokens=0)), service_tier=None)..
[13:53:17] Retrying in 0.0 seconds as it raised MalformedMessageError: Unexpected shape of LiteLLM model response ModelResponse(id='chatcmpl-A8nX6ZPXYLdCL1j8StAw9aU57zOoG',
choices=[Choices(finish_reason='stop', index=0, message=Message(content="I am currently unable to retrieve papers on AI regulation in China and the EU. However, I can provide a general overview based
on existing knowledge.\n\nAI Regulation in China:\n1. Centralized Approach: China's AI regulation is characterized by a top-down, centralized approach. The government plays a significant role
in setting AI policies and regulations.\n2. Focus on National Security and Social Stability: Regulations often emphasize national security, social stability, and the promotion of AI for economic
development.\n3. Data Privacy: While there are data privacy laws, such as the Personal Information Protection Law (PIPL), enforcement and scope can differ from Western standards.\n4. Ethical
Guidelines: China has issued ethical guidelines for AI, focusing on ensuring AI is controllable and trustworthy.\n\nAI Regulation in the EU:\n1. Human-Centric and Ethical Approach: The EU's
approach is more human-centric, focusing on ethical AI and protecting fundamental rights.\n2. GDPR and Data Privacy: The General Data Protection Regulation (GDPR) is a cornerstone of EU data
privacy, influencing AI regulation by emphasizing user consent and data protection.\n3. AI Act Proposal: The EU has proposed the AI Act, which categorizes AI systems based on risk and sets
requirements accordingly, aiming to ensure safety and fundamental rights.\n4. Decentralized and Collaborative: The EU's regulatory approach involves collaboration among member states and
stakeholders, reflecting a more decentralized process compared to China.\n\nIf you need more detailed or specific information, I recommend consulting recent academic papers or official regulatory
documents from both regions.", role='assistant', tool_calls=None, function_call=None))], created=1726660392, model='gpt-4o-2024-08-06', object='chat.completion', system_fingerprint='fp_143bb8492c',
usage=Usage(completion_tokens=320, prompt_tokens=873, total_tokens=1193, completion_tokens_details=CompletionTokensDetails(reasoning_tokens=0)), service_tier=None)..
[13:53:18] Agent <aviary.tools.utils.ToolSelector object at 0x7fd42ea964d0> failed.
╭───────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────╮
│ /home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/main.py:210 in run_aviary_agent │
│ │
│ 207 │ │ │ │
│ 208 │ │ │ while not done: │
│ 209 │ │ │ │ agent_state.messages += obs │
│ ❱ 210 │ │ │ │ for attempt in Retrying( │
│ 211 │ │ │ │ │ stop=stop_after_attempt(5), │
│ 212 │ │ │ │ │ retry=retry_if_exception_type(MalformedMessageError), │
│ 213 │ │ │ │ │ before_sleep=before_sleep_log(logger, logging.WARNING), │
│ │
│ /home/jay/paperqa/lib/python3.11/site-packages/tenacity/init.py:443 in iter │
│ │
│ 440 │ │ │
│ 441 │ │ retry_state = RetryCallState(self, fn=None, args=(), kwargs={}) │
│ 442 │ │ while True: │
│ ❱ 443 │ │ │ do = self.iter(retry_state=retry_state) │
│ 444 │ │ │ if isinstance(do, DoAttempt): │
│ 445 │ │ │ │ yield AttemptManager(retry_state=retry_state) │
│ 446 │ │ │ elif isinstance(do, DoSleep): │
│ │
│ /home/jay/paperqa/lib/python3.11/site-packages/tenacity/init.py:376 in iter │
│ │
│ 373 │ │ self._begin_iter(retry_state) │
│ 374 │ │ result = None │
│ 375 │ │ for action in self.iter_state.actions: │
│ ❱ 376 │ │ │ result = action(retry_state) │
│ 377 │ │ return result │
│ 378 │ │
│ 379 │ def _begin_iter(self, retry_state: "RetryCallState") -> None: # noqa │
│ │
│ /home/jay/paperqa/lib/python3.11/site-packages/tenacity/init.py:418 in exc_check │
│ │
│ 415 │ │ │ │ fut = t.cast(Future, rs.outcome) │
│ 416 │ │ │ │ retry_exc = self.retry_error_cls(fut) │
│ 417 │ │ │ │ if self.reraise: │
│ ❱ 418 │ │ │ │ │ raise retry_exc.reraise() │
│ 419 │ │ │ │ raise retry_exc from fut.exception() │
│ 420 │ │ │ │
│ 421 │ │ │ self._add_action_func(exc_check) │
│ │
│ /home/jay/paperqa/lib/python3.11/site-packages/tenacity/init.py:185 in reraise │
│ │
│ 182 │ │
│ 183 │ def reraise(self) -> t.NoReturn: │
│ 184 │ │ if self.last_attempt.failed: │
│ ❱ 185 │ │ │ raise self.last_attempt.result() │
│ 186 │ │ raise self │
│ 187 │ │
│ 188 │ def str(self) -> str: │
│ │
│ /usr/lib/python3.11/concurrent/futures/_base.py:449 in result │
│ │
│ 446 │ │ │ │ if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: │
│ 447 │ │ │ │ │ raise CancelledError() │
│ 448 │ │ │ │ elif self._state == FINISHED: │
│ ❱ 449 │ │ │ │ │ return self.__get_result() │
│ 450 │ │ │ │ │
│ 451 │ │ │ │ self._condition.wait(timeout) │
│ 452 │
│ │
│ /usr/lib/python3.11/concurrent/futures/_base.py:401 in __get_result │
│ │
│ 398 │ def __get_result(self): │
│ 399 │ │ if self._exception: │
│ 400 │ │ │ try: │
│ ❱ 401 │ │ │ │ raise self._exception │
│ 402 │ │ │ finally: │
│ 403 │ │ │ │ # Break a reference cycle with the exception in self._exception │
│ 404 │ │ │ │ self = None │
│ │
│ /home/jay/paperqa/lib/python3.11/site-packages/paperqa/agents/main.py:217 in run_aviary_agent │
│ │
│ 214 │ │ │ │ │ reraise=True, │
│ 215 │ │ │ │ ): │
│ 216 │ │ │ │ │ with attempt: # Retrying if ToolSelector fails to select a tool │
│ ❱ 217 │ │ │ │ │ │ action = await agent(agent_state.messages, tools) │
│ 218 │ │ │ │ agent_state.messages = [*agent_state.messages, action] │
│ 219 │ │ │ │ if on_agent_action_callback: │
│ 220 │ │ │ │ │ await on_agent_action_callback(action, agent_state) │
│ │
│ /home/jay/paperqa/lib/python3.11/site-packages/aviary/tools/utils.py:64 in call │
│ │
│ 61 │ │ │ len(model_response.choices) != 1 │
│ 62 │ │ │ or model_response.choices[0].finish_reason != "tool_calls" │
│ 63 │ │ ): │
│ ❱ 64 │ │ │ raise MalformedMessageError( │
│ 65 │ │ │ │ f"Unexpected shape of LiteLLM model response {model_response}." │
│ 66 │ │ │ ) │
│ 67 │ │ usage = model_response.usage │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
MalformedMessageError: Unexpected shape of LiteLLM model response ModelResponse(id='chatcmpl-A8nXBPXbvmzqjhA3OEx6KdPp8N0GY', choices=[Choices(finish_reason='stop', index=0, message=Message(content='It
seems there is an issue with retrieving papers at the moment. Unfortunately, I am unable to proceed with gathering evidence or generating an answer without access to relevant papers. You might want to
try again later or consult other resources for information on AI regulation differences between China and the EU.', role='assistant', tool_calls=None, function_call=None))], created=1726660397,
model='gpt-4o-2024-08-06', object='chat.completion', system_fingerprint='fp_143bb8492c', usage=Usage(completion_tokens=57, prompt_tokens=873, total_tokens=930,
completion_tokens_details=CompletionTokensDetails(reasoning_tokens=0)), service_tier=None).
Answer: `

It seems as if I am already hitting the OpenAI token per minute tpm rate limit while indexing the papers? it is only about 20 papers and i dont get why it is asking for 3000 tokens per minute so often and so early? What am I missing?

cheers
justus

Answered by jamesbraza

Hi @derspotter , we have an outstanding request in #377 to make this easier to resolve. Stay tuned there 👌

View full answer

Replies: 1 comment

jamesbraza
Sep 19, 2024
Maintainer

Hi @derspotter , we have an outstanding request in #377 to make this easier to resolve. Stay tuned there 👌

0 replies

Answer selected by jamesbraza

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment