Support for other LLMs?

[munishm]

Hi, Is there any roadmap to support other open source LLM's? If there is any documentation already in place, please share.

[Fei-Guo]

No roadmap for exactly what models will be added yet. The idea is to collect user feedback or based on model popularity for a priority list. Technically, we will optimize dev workflow for adding new models so in the end, it can be a community driven process.

[thegovind]

So, if I want to host phi-2, do I have to fork this repo and configure it myself?

[Fei-Guo]

I think we can support phi-2 formally in kaito. We are working on a serial of documents about how to propose/contribute to add new models in Kaito. Stay tuned.

[Fei-Guo]

https://github.com/Azure/kaito/blob/main/docs/How-to-add-new-models.md

Here is the workflow we designed for new model onboarding.