How to add tool calls to the ChatHistory?

[ColtonBehannon]

I have a chat bot configured as a kernel function. When it is invoked I add the user input to the chat history with chat_history.add_user_message() and the chat bot response with chat_history.add_assistant_message(). I would like to include the tool calls in the chat history as well but am not sure how to go about it. I see that the ChatHistory object has .add_tool_message() however I am not sure how to extract the tool calls to pass them here.

Even more preferable to manually adding them here would be if the tool calls are automatically added to the chat history when they are invoked. This dev blog mentions that when we enable auto tool calling that, "The result of the function calls are added to the chat history with the role tool_call." However, when I run the sample associated with this blog I am not seeing them added to the chat history either.

Any help would be greatly appreciated.

The relevant section of my code can be seen below:

system_message = """
You are a chat bot. Your purpose is to assist the user in executing a plan that was devised to gather information on a topic.
The plan will be conveyed to you and you are to use the plugins provided to you as needed to execute the plan
"""

prompt_template_config = PromptTemplateConfig(
    template="""{{$chat_history}}{{$user_input}}""",
    name="chat",
    template_format="semantic-kernel",
    input_variables=[
        InputVariable(name="chat_history", description="The history of the conversation", is_required=True),
        InputVariable(name="user_input", description="The user input", is_required=True),
    ],
)

# Configure chat
### Establish prompt execution settings
req_settings = kernel.get_prompt_execution_settings_from_service_id(service_id=service_id)
req_settings.max_tokens = 2000
req_settings.temperature = 0.1
req_settings.top_p = 0.8
req_settings.function_call_behavior = FunctionCallBehavior.EnableFunctions(
    auto_invoke=True, filters={"excluded_plugins": ["ChatBot"]}
)

chat_history = ChatHistory(system_message=system_message)

### Create the chat function
chat_function = kernel.add_function(
    function_name="Chat",
    plugin_name="ChatBot",
    prompt_execution_settings=req_settings,
    prompt_template_config=prompt_template_config
)
arguments = KernelArguments(settings=req_settings)

options = FunctionCallingStepwisePlannerOptions(
    max_iterations=10, max_tokens=400
)

    
async def create_plan(user_input) -> bool:
    chat_history.add_user_message(user_input)
    planner = FunctionCallingStepwisePlanner(service_id=service_id, options=options)
    llm_generated_plan = await planner._generate_plan(
        question=user_input, kernel=kernel, arguments=KernelArguments()
    )
    print(f"Proposed plan:\n", llm_generated_plan)
    user_decision = input("Do you accept the proposed plan? Respond 'y' or 'n'.\nUser:> ")
    if user_decision == 'y':
        chat_history.add_assistant_message(llm_generated_plan)
        return llm_generated_plan
    elif user_decision == 'n':
        user_edited_plan = input("Please make edits to the plan and resubmit.\nUser:> ")
        chat_history.add_assistant_message(user_edited_plan)
        return user_edited_plan

async def chat(iteration: int) -> bool:
    try:
        user_input = input("User:> ")
    except KeyboardInterrupt:
        print("\n\nExiting chat...")
        return False
    except EOFError:
        print("\n\nExiting chat...")
        return False
    if user_input == "exit":
        print("\n\nExiting chat...")
        return False
    
    if iteration == 0:
        plan = await create_plan(user_input)
        proceed_request = "Proceed with the proposed plan."
        arguments['chat_history'] = chat_history
        arguments['user_input'] = proceed_request
        chat_response = await kernel.invoke(
            function=chat_function,
            arguments=arguments
        )
        print(f"ChatBot:> {chat_response}")
        chat_history.add_user_message(proceed_request)
        chat_history.add_assistant_message(str(chat_response))
    else:
        arguments['chat_history'] = chat_history
        arguments['user_input'] = user_input
        chat_response = await kernel.invoke(
            function=chat_function,
            arguments=arguments
        )
        print(f"ChatBot:> {chat_response}")
        chat_history.add_user_message(user_input)
        chat_history.add_assistant_message(str(chat_response))
        
    return True
            

async def main() -> None:
    print(
        "Welcome to the chat bot!\
        \n  Type 'exit' to exit.\
        \n  Ask any questions to get started."
    )
    chatting = True
    i = 0
    while chatting:
        chatting = await chat(iteration=i)
        i+=1


if __name__ == "__main__":
    asyncio.run(main())

[sophialagerkranspandey]

Looping in @alliscode and @moonbox3 for next steps

[moonbox3]

Hi @ColtonBehannon, apologies for the delay. When you get the chat_response back after function invocation, you can dig into its metadata to the messages. In there, you see the tool calls that were made. Does this help you?

[moonbox3]

Hi @ColtonBehannon, you'll want to grab the FunctionCallContent from the ChatMessageContent that has the tool_calls Finish Reason. This FunctionCallContent contains the metadata you'd need related to the tool call. From there you can call chat_history.add_tool_message([FunctionCallContent]).

[ColtonBehannon]

Perfect, thank you very much!

[noeltupas]

FunctionCallContent would be great but it seems to be susceptible to removal:

Could someone please advise on another way that is as easily accessible as FunctionCallContent? I asked because mining it out of the metadata is a bit burdensome.

[moonbox3]

tagging @markwallace-microsoft for visibility into the dotnet FunctionCallContent stability.

[noeltupas]

@markwallace-microsoft Could you please provide some insight into using FunctionCallContent and recommendations for/against it?