Is Kernel API overcomplicated?

[rosieks]

I'm following Semantic Kernel for about half year if not longer I have impression that design of the framework is messy and hard to understand.

1. What is Kernel. There is Kernel object but what's the responsibility of that?
   1.1. It has reference IServiceProvider but what for? For that reasons we have IServiceProvider and we can inject specific AI service directly to our class without having reference to Kernel.
   1.2. It stores plugins. But do they work with all AI Services or only with IChatCompletion and only for specific implementations.
   1.3. If I register plugins in Kernel, then I will call function to get IChatCompletion service it's still not aware of those plugins. I have to pass Kernel back as parameter. The same if I create function from prompt and want to invoke it. Despite it was created by this Kernel.
   1.4. I can use Kernel to create function from prompt. But not, wait a minute. It only takes it to get logger. If I want invoke that function I have to pass Kernel again as parameter.
   1.5. In general Kernel looks like some kind of god class that mix multiple responsibilities (services factory, filters pipeline, plugins register class that provides reference to logger) and maybe should be splited.
2. IChatCompletion looks like unnecessary abstraction because when we want to invoke AI we have to provide specific (or at least that what we see in most samples) PromptExecutionSettings with model specific configuration.
3. Dependency Injection (especially in documentation) looks like added 5 minutes before release. Expecting user to call DI container by himself doesn't seem to me like supporting DI but rather showing workaround how to deal with that.

I don't know if that's only mine impression or there are others who think the same.

[rosieks]

Just to add more. I just noticed that chatCompletion.GetChatMessageContentAsync mutate state of parameters - it adds content to ChatHistory.

[akordowski]

@rosieks I have the same impression. You already mentioned some valid points, but also for me there is a lot where it is hard for me to understand the design choices and how the things work together. Nevertheless, it is a fascinating project.

[markwallace-microsoft]

Hi @rosieks
thanks for the questions, let me explain what our thinking is.

1. What is the Kernel? We discussed this a lot as a team and did consider eliminating this. In the end we decide to keep it and have it as the central component in our architecture. So the Kernel is typically a transient object that maintains state that can be used during the processing of a request e.g. functions to use for either prompt rendering or model function calling, the filters to execute to control execution, ... We keep the lifecycle of the Kernel seperate to the lifecycle of other components e.g. if I import a plugin from an Open API definition I only want to do this once and be able to reuse these functions for multiple requests with multiple Kernel instances. We do add some methods to the Kernel to help with discoverability and that may be causing some confusion.
2. IChatCompletion is our abstraction for a chat completion endpoint and is provided to allow you to write code that can switch between different endpoints e.g. from OpenAI to Azure Open AI to Mistal. The PromptExecutionSettings allow you to customise execution for a specific prompt e.g. for different prompt I may want different Temperature settings or function calling configuration.
3. We have you issue on our Dependency Injection support and we are planning to improve the documentation. We have had positive feedback about this aspect of our design but we are eager to make things better based on constructive feedback.
4. Having chatCompletion.GetChatMessageContentAsync add content to ChatHistory was intentional e.g. during a function calling loop there may be multiple round trips to the LLM.

Thanks again for taking the time to provide your feedback, I'll share this with the team.

[rosieks]

Thank you for answer, I really like concept of Semantic Kernel. I think the documentation can't keep up with the pace of adding new pieces of code. That together with unusual design decisions leads to a lot of confusion.
It would be really great if you catch up with documentation for core functionality. Explain how do you see Kernel, how it was designed, what are best practices to use it, what are the reasons (use cases) it is mutable.
Personally I think that mutability of Kernel class generates most of confusion.

[dmytrostruk]

@rosieks Regarding Dependency Injection, I've already implemented several web applications with Semantic Kernel and my DI setup looks like this all the time:

var builder = WebApplication.CreateBuilder(args);

// Add Semantic Kernel
builder.Services.AddKernel();
builder.Services.AddOpenAIChatCompletion(openAIOptions.ChatModelId, openAIOptions.ApiKey);

When I want to include some plugins, I just import them in place where I want them to be used, for example in some service, where I inject a Kernel:

this._kernel.ImportPluginFromType<TimePlugin>();
this._kernel.ImportPluginFromType<WeatherPlugin>();

Taking into account, that my Kernel is transient, it's always injected to my service without Plugins, which allows me to share with AI only those plugins I want to use in specific place of my application, but not all of them. Because I may have plugins related to different domain entities in my app, so I definitely want to separate them. I would avoid registering a lot of plugins and Kernel as singletons and use them in all places of my app without any isolation. In SK codebase, there is no simple way to register Kernel and Plugin collection as singletons, because I'm not sure it's the best practice, and as far as I understand, your complaint about DI is related to singleton and scoped lifecycles. In Kernel, you can mutate Plugin Collection. If Plugin Collection will be a singleton, you may end up with two different Kernel instances that will mutate the same Plugin Collection, which may lead to unexpected behavior. That's why extension method AddKernel registers Plugin Collection and Kernel as transient.

It would be really helpful if you could share some information about your use case to better understand the purpose of registering Kernel and Plugin collection as scoped/singleton, so we can prioritize it. If you are interested in sharing some state and scoped/singleton lifecycle works better for you, you can still achieve that with more complex registration logic, but so far it doesn't look like a common scenario, and current extension methods cover most of the cases. If it's not true, we will definitely add more helpers to simplify DI if needed.

Regarding a responsibility of Kernel, I think it's possible to work without it, but if you need Plugins, Chat/Text completion service, Prompt Template Engines, Middleware, Telemetry etc, you can do that without Kernel, but you will need to pass these things around your application as a bundle and manage it on your side. I think Kernel works like a container, which you can re-use multiple times in your app, so in business logic you don't need to operate with these multiple components and mix this logic, you just need one Kernel. So, it works like DI container plus entrypoint to AI at the same time. Currently, we mostly focus on IChatCompletionService because it's the most popular AI capability, but ideally Kernel will be capable of working with text-to-image/text-to-audio services, vector databases etc.

I hope that makes sense and thank you for feedback!

[rosieks]

Thanks for explanation how you see it. I really apricate that. I think that mutable nature Kernel introduce this confusion. Most developers (I think) is used to that if you inject something into your class it's preconfigured and ready to use. In your example I see it's opposite and you have to add plugins in the class that consume Kernel. I just wonder how do you manage which filter to use or not. Unfortunately documentation doesn't explain how to register filter. If filters are registered on DI container level and plugins on instance level then it creates another confusion/inconsistency why?

Regarding lifecycle of plugins I don't mind having them as transient. The only thing that I don't like is that you don't cache plugin metadata. On every request you import plugin from scratch, you travers with reflection all methods to find witch one is KernelFunction. I think it's waste of resources.

I would really love to see some real web application sample with documentation how it works. But something easy and not so much complicated like Copilot chat in the past.

[rosieks]

One more thing regarding registering plugins. If that's the way how it suppose to be used then you should show it in documentation regarding dependency injection. Right now documentation shows that you register KernelPluginCollection as singleton :)

[dmytrostruk]

Yes, the documentation will be updated.