Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Logprobs in MLC Batch Serving #82

Merged
merged 42 commits into from
Jan 31, 2024
Merged

Enable Logprobs in MLC Batch Serving #82

merged 42 commits into from
Jan 31, 2024

Commits on Jan 25, 2024

  1. Squashed commit for logprobs implementation. 

    Co-authored-by: Valery Chernov <[email protected]>
Co-authored-by: Ilya Kozulin <[email protected]>
    @zxybazh @vvchernov @Ailurus1
    3 people committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    ab47b41 View commit details
    Browse the repository at this point in the history

  2. fix None check

    @vvchernov @zxybazh
    vvchernov authored and zxybazh committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    86f6fa1 View commit details
    Browse the repository at this point in the history

  3. Change detokenization to using token ids.

    @zxybazh
    zxybazh committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    9a29650 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2024

  1. Fix wrong usage of token ids. Remove logging.

    @zxybazh
    zxybazh committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    012388d View commit details
    Browse the repository at this point in the history

  2. extend benchmarks for logprobs

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    db31164 View commit details
    Browse the repository at this point in the history

  3. fix test without logprobs

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    be81755 View commit details
    Browse the repository at this point in the history

  4. clean code

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    e8ec3fc View commit details
    Browse the repository at this point in the history

  5. black format engine_common.py

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    49187f5 View commit details
    Browse the repository at this point in the history

  6. logprobs is strictly bool, top_logprobs is int

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    013ed5a View commit details
    Browse the repository at this point in the history

  7. refactor logprob info collection to not reduce performance

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    79ec413 View commit details
    Browse the repository at this point in the history

  8. quick fix for check

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    fca1a6f View commit details
    Browse the repository at this point in the history

  9. review fix

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    675b631 View commit details
    Browse the repository at this point in the history

  10. fix list index out of range

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    18f80fa View commit details
    Browse the repository at this point in the history

  11. rollback after rebase

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    29ea525 View commit details
    Browse the repository at this point in the history

  12. test

    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    aa99322 View commit details
    Browse the repository at this point in the history

  13. Merge pull request #7 from Deelvin/vc/benchmark 

    Update benchmark tests and fixes after review
    @vvchernov
    vvchernov committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    8fa785e View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. Squashed commit for logprobs implementation. 

    Co-authored-by: Valery Chernov <[email protected]>
Co-authored-by: Ilya Kozulin <[email protected]>
    @zxybazh @vvchernov @Ailurus1
    3 people committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    d57b197 View commit details
    Browse the repository at this point in the history

  2. fix None check

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    7995c84 View commit details
    Browse the repository at this point in the history

  3. Change detokenization to using token ids.

    @zxybazh @vvchernov
    zxybazh authored and vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    ae3fc5b View commit details
    Browse the repository at this point in the history

  4. Fix wrong usage of token ids. Remove logging.

    @zxybazh @vvchernov
    zxybazh authored and vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    0cb036f View commit details
    Browse the repository at this point in the history

  5. extend benchmarks for logprobs

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    ed51e7d View commit details
    Browse the repository at this point in the history

  6. fix test without logprobs

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    ff17ae2 View commit details
    Browse the repository at this point in the history

  7. clean code

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    f5e4339 View commit details
    Browse the repository at this point in the history

  8. black format engine_common.py

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a3f6e8b View commit details
    Browse the repository at this point in the history

  9. logprobs is strictly bool, top_logprobs is int

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c54a410 View commit details
    Browse the repository at this point in the history

  10. refactor logprob info collection to not reduce performance

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    379d991 View commit details
    Browse the repository at this point in the history

  11. quick fix for check

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    58bac8f View commit details
    Browse the repository at this point in the history

  12. review fix

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    7de8d88 View commit details
    Browse the repository at this point in the history

  13. fix list index out of range

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    661fa18 View commit details
    Browse the repository at this point in the history

  14. rollback after rebase

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    6662a65 View commit details
    Browse the repository at this point in the history

  15. test

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    970d7f8 View commit details
    Browse the repository at this point in the history

  16. small fix

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c58d69c View commit details
    Browse the repository at this point in the history

  17. rename for the sake of clarity

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    ebae200 View commit details
    Browse the repository at this point in the history

  18. some fixes with cpu-gpu tensor copying

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    b2863d5 View commit details
    Browse the repository at this point in the history

  19. refactor logprob pass to calculate

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    57b3a35 View commit details
    Browse the repository at this point in the history

  20. remove excess deps for token detokenization

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4e29403 View commit details
    Browse the repository at this point in the history

  21. small clean

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a9157b9 View commit details
    Browse the repository at this point in the history

  22. small clean

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    39efb61 View commit details
    Browse the repository at this point in the history

  23. return None instead of list of Nones

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    601e68d View commit details
    Browse the repository at this point in the history

  24. resolve conflicts

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4f9241b View commit details
    Browse the repository at this point in the history

  25. fix mypy

    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    7ec21a7 View commit details
    Browse the repository at this point in the history

  26. Merge pull request #8 from Deelvin/vc/perf 

    Fix performance reduction
    @vvchernov
    vvchernov committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    7aa60ed View commit details
    Browse the repository at this point in the history