Python: Bug: The 'get_chat_message_contents' returns with a tool_calls message without calling the plugin's function(s) (intermittently)

[samelhousseini]

Describe the bug
This is an intermittent problem and hard to reproduce with exact steps.

To Reproduce
Steps to reproduce the behavior:
The "chat_completion.get_chat_message_contents" function call sometimes just returns with a tool_calls message but without calling the plugin's functions described in the tool_calls. The chat history indicates that the last message added is a "tool_calls" but no function result message is added afterwards.

Expected behavior
Since there's a tool calls, the expected behavior is that the function call workflow proceeds as expected with a function call and a function result, but then all I get is a tool_calls message from "chat_completion.get_chat_message_contents"

result = (await self.chat_completion.get_chat_message_contents(
                chat_history=self.history,
                settings=self.execution_settings,
                kernel=self.kernel,
                arguments=KernelArguments(),
                temperature=0.2
            ))[0]

When inspecting the results above, the get_chat_message_contents is returning without calling the plugin's functions:

Screenshots
Screenshot below showing that the call to get_chat_message_contents just returned with an empty string, because the ChatMessageContent is a function call "tool_calls".

print("Assistant > " + str(result))

And that the call to the filter did not happen.

Platform

• SK: semantic-kernel 1.8.3 python package
• OS: Windows
• IDE: VS Code
• Language: Python

Update:

The following block of code is a temp fix of this problem:

        counter = 0

        while True:
            print(f"-------------- Attempting response {counter}")
            result = (await self.chat_completion.get_chat_message_contents(
                chat_history=self.history,
                settings=self.execution_settings,
                kernel=self.kernel,
                arguments=KernelArguments(),
                temperature=0.2
            ))[0]

            if result.finish_reason == 'tool_calls':
                self.history.add_message(result)
                function_results = []

                for item in result.items:
                    content = await self.kernel.plugins[item.plugin_name].functions[item.function_name].method(**json.loads(item.arguments))
                    print("Content from Function Call - ", content)
                    function_results.append(
                        FunctionResultContent(
                            id=item.id, name=item.name, result=str(content)
                        )
                    )                    
                    
                self.history.add_message(
                    ChatMessageContent(
                        role = AuthorRole.TOOL,
                        items = function_results
                    )
                )
            else:
                break

            counter += 1

            if counter > 25: break

        # Add the message from the agent to the chat history
        self.history.add_message(result)

[eavanvalkenburg]

@samelhousseini what are the settings you are using?

[samelhousseini]

Hi @eduard van ***@***.***>, I believe you meant the execution settings? Please find below. self.execution_settings = AzureChatPromptExecutionSettings(tool_choice="auto") self.execution_settings.function_choice_behavior = FunctionChoiceBehavior.Auto() Many thanks Samer

…

________________________________ From: Eduard van Valkenburg ***@***.***> Sent: Monday, September 16, 2024 6:11 PM To: microsoft/semantic-kernel ***@***.***> Cc: Mention ***@***.***>; Author ***@***.***> Subject: Re: [microsoft/semantic-kernel] Python: Bug: The 'get_chat_message_contents' returns with a tool_calls message without calling the plugin's function(s) (intermittently) (Issue #8712) @samelhousseini<https://github.com/samelhousseini> what are the settings you are using? — Reply to this email directly, view it on GitHub<#8712 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/AZ2RAUXNZ3WJHMNQB7ODC3TZW3RJXBFKMF2HI4TJMJ2XIZLTS2BKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVI2TEMBVGAYTIMJVHCSG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRVGI4DSNJTHEYTCOFENZQW2ZNJNBQXGX3MMFRGK3ECUV3GC3DVMWVDOMBWGM4DMMZYGU4KI3TBNVS2S2DBONPWYYLCMVWKY43VMJVGKY3UL52HS4DFVREXG43VMVBW63LNMVXHJJTUN5YGSY3TSWBKI5DZOBS2U4TFOBXXG2LUN5ZHTJLWMFWHKZNJGYYDOMRYHEYTQNMCUR2HS4DFUVUXG43VMWSXMYLMOVS2UMRVGI2DOOJSGE2DDAVEOR4XAZNFNRQWEZLMUV3GC3DVMWVDKMRQGUYDCNBRGU4IFJDUPFYGLJLMMFRGK3FFOZQWY5LFVI2TEOBZGUZTSMJRHCBKI5DZOBS2K3DBMJSWZJLWMFWHKZNKG4YDMMZYGYZTQNJYU52HE2LHM5SXFJTDOJSWC5DF>. You are receiving this email because you were mentioned. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.