New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Client Added for Predibase #40

Open

bhaba-ranjan wants to merge 2 commits into ray-project:main from bhaba-ranjan:predibase-llm-inf-test

bhaba-ranjan commented Mar 23, 2024 •

edited

Loading

Hello,

This pull request adds Ray Client for Predibase.
Predibase is an open-source LLM inference server. It can serve multiple LoRA adapters, in a single deployment.
The pull request targets /generate_stream endpoint which streams token.
API spec can be found here: Predibase Swagger

bhaba-ranjan added 2 commits

March 22, 2024 20:00


          feat: added client for predibase

e37ad35


          feat: added client for predibase

f5223fb

bhaba-ranjan changed the title ~~feat: Client Added for Predibase~~ feat: Client Added for Predibase and Hugging Face

bhaba-ranjan changed the title ~~feat: Client Added for Predibase and Hugging Face~~ feat: Client Added for Predibase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet