Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Pagination to Logs #1012

Closed
wants to merge 2 commits into from
Closed

Add Pagination to Logs #1012

wants to merge 2 commits into from

Conversation

NolanTrem
Copy link
Collaborator

@NolanTrem NolanTrem commented Aug 30, 2024

Checking in for now—will revisit, but the logs that are returned do not seem to be correct.

INFO: 127.0.0.1:51443 - "GET /v2/logs?limit=5 HTTP/1.1" 200 OK
2024-08-30 11:08:37,509 - INFO - core.main.services.management_service - alogs called with offset=0, limit=3
2024-08-30 11:08:37,514 - INFO - core.main.services.management_service - get_info_logs returned 3 entries and they are: [RunInfoLog(run_id=UUID('2e305f25-ad1b-57f2-a0f2-b3bf4cb962cf'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 37), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('1a8ba171-7044-5703-9519-add61dcdcb44'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 3, 44), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('8c6d76ca-42ff-572e-857e-fbb49f98de8d'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 3, 18), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220'))]
2024-08-30 11:08:37,522 - INFO - core.main.services.management_service - get_logs returned 0 entries and they are: []
INFO: 127.0.0.1:51538 - "GET /v2/logs?limit=3 HTTP/1.1" 200 OK
2024-08-30 11:08:43,059 - INFO - core.main.services.management_service - alogs called with offset=0, limit=3
2024-08-30 11:08:43,060 - INFO - core.main.services.management_service - get_info_logs returned 3 entries and they are: [RunInfoLog(run_id=UUID('b6928795-f58f-569e-821f-8c84fba45bac'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 43), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('2e305f25-ad1b-57f2-a0f2-b3bf4cb962cf'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 37), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('1a8ba171-7044-5703-9519-add61dcdcb44'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 3, 44), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220'))]
2024-08-30 11:08:43,061 - INFO - core.main.services.management_service - get_logs returned 0 entries and they are: []
INFO: 127.0.0.1:51543 - "GET /v2/logs?limit=3 HTTP/1.1" 200 OK
2024-08-30 11:08:49,191 - INFO - core.main.services.management_service - alogs called with offset=0, limit=3
2024-08-30 11:08:49,192 - INFO - core.main.services.management_service - get_info_logs returned 3 entries and they are: [RunInfoLog(run_id=UUID('5241de1c-064b-5834-b76f-28c6493e01b5'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 49), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('b6928795-f58f-569e-821f-8c84fba45bac'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 43), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('2e305f25-ad1b-57f2-a0f2-b3bf4cb962cf'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 37), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220'))]
2024-08-30 11:08:49,193 - INFO - core.main.services.management_service - get_logs returned 0 entries and they are: []
INFO: 127.0.0.1:51547 - "GET /v2/logs?offset=0&limit=3 HTTP/1.1" 200 OK
2024-08-30 11:09:03,819 - INFO - core.main.services.management_service - alogs called with offset=1, limit=3
2024-08-30 11:09:03,820 - INFO - core.main.services.management_service - get_info_logs returned 3 entries and they are: [RunInfoLog(run_id=UUID('5241de1c-064b-5834-b76f-28c6493e01b5'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 49), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('b6928795-f58f-569e-821f-8c84fba45bac'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 43), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('2e305f25-ad1b-57f2-a0f2-b3bf4cb962cf'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 37), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220'))]
2024-08-30 11:09:03,822 - INFO - core.main.services.management_service - get_logs returned 0 entries and they are: []
INFO: 127.0.0.1:51557 - "GET /v2/logs?offset=1&limit=3 HTTP/1.1" 200 OK
2024-08-30 11:09:23,461 - INFO - core.main.services.management_service - alogs called with offset=4, limit=3
2024-08-30 11:09:23,462 - INFO - core.main.services.management_service - get_info_logs returned 3 entries and they are: [RunInfoLog(run_id=UUID('2e305f25-ad1b-57f2-a0f2-b3bf4cb962cf'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 8, 37), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('1a8ba171-7044-5703-9519-add61dcdcb44'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 3, 44), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220')), RunInfoLog(run_id=UUID('8c6d76ca-42ff-572e-857e-fbb49f98de8d'), run_type='MANAGEMENT', timestamp=datetime.datetime(2024, 8, 30, 18, 3, 18), user_id=UUID('2acb499e-8428-543b-bd85-0d9098718220'))]
2024-08-30 11:09:23,463 - INFO - core.main.services.management_service - get_logs returned 0 entries and they are: []
INFO: 127.0.0.1:51565 - "GET /v2/logs?offset=4&limit=3 HTTP/1.1" 200 OK


🚀 This description was created by Ellipsis for commit 73dfae6

Summary:

This PR adds pagination to logs retrieval across CLI, API, and logging components, and updates GitHub Actions workflows for architecture-specific builds.

Key points:

  • Added pagination to logs retrieval in py/cli/commands/server.py by introducing offset and limit options.
  • Updated py/core/base/logging/run_logger.py to support pagination in get_info_logs method.
  • Modified py/core/main/api/routes/management/base.py to include offset and limit in /logs endpoint.
  • Enhanced py/core/main/services/management_service.py to handle pagination in alogs method.
  • Adjusted py/sdk/server.py to pass pagination parameters in logs method.
  • Split build jobs by architecture in .github/workflows/build-main.yml.
  • Changed runner configuration in .github/workflows/py-ci-cd.yml.

Generated with ❤️ by ellipsis.dev

Copy link

codecov bot commented Aug 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Flag Coverage Δ
python 75.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@emrgnt-cmplxty emrgnt-cmplxty changed the base branch from main to dev August 30, 2024 23:56
@emrgnt-cmplxty emrgnt-cmplxty marked this pull request as ready for review September 4, 2024 02:53
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 73dfae6 in 26 seconds

More details
  • Looked at 403 lines of code in 7 files
  • Skipped 0 files when reviewing.
  • Skipped posting 5 drafted comments based on config settings.
1. py/cli/commands/server.py:42
  • Draft comment:
    Consider setting a default value for offset that aligns with backend expectations, such as 0, to avoid potential issues if the backend does not handle None values.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces pagination with offset and limit parameters, but the default values for these parameters are set to None in the CLI command, which might lead to unexpected behavior if not handled properly in the backend. It's better to set them to default values that align with the backend expectations.
2. py/cli/commands/server.py:45
  • Draft comment:
    Consider setting a default value for limit that aligns with backend expectations, such as 100, to avoid potential issues if the backend does not handle None values.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces pagination with offset and limit parameters, but the default values for these parameters are set to None in the CLI command, which might lead to unexpected behavior if not handled properly in the backend. It's better to set them to default values that align with the backend expectations.
3. py/core/base/logging/run_logger.py:602
  • Draft comment:
    Consider adding offset and limit parameters to get_logs for consistency with get_info_logs and to support pagination.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR modifies the get_info_logs method to include offset and limit parameters, but the get_logs method still uses a fixed limit_per_run value. It would be more consistent to allow pagination for get_logs as well.
4. py/core/base/logging/run_logger.py:756
  • Draft comment:
    Consider adding offset and limit parameters to get_logs for consistency with get_info_logs and to support pagination.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR modifies the get_info_logs method to include offset and limit parameters, but the get_logs method still uses a fixed limit_per_run value. It would be more consistent to allow pagination for get_logs as well.
5. py/core/main/services/management_service.py:69
  • Draft comment:
    Consider adding offset and limit parameters to get_logs for consistency with get_info_logs and to support pagination.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR modifies the get_info_logs method to include offset and limit parameters, but the get_logs method still uses a fixed limit_per_run value. It would be more consistent to allow pagination for get_logs as well.

Workflow ID: wflow_7EWTby6KKvt7zci0


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@NolanTrem NolanTrem closed this Sep 4, 2024
@NolanTrem NolanTrem deleted the Nolan/Logs branch September 6, 2024 01:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant