I am very interested in your suggestions because I also encounter the same problem as you mentioned. Can you state in detail the structure you built for query messages history (user queries along with optimized query response from OpenAI)?

Currently in query messages history we keep only user questions + optimized search query from OpenAI endpoint. Example (I wrote it myself now, didn't copy it from actual queries):

[
  {
    "role": "user"
    "content": "what is abc?"
  },
  {
    "role": "assistant"
    "content": "abc definition"
  },
  {
    "role": "user"
    "content": "what is def?"
  },
  {
    "role": "assistant"
    "content": "def definition"
  },
  {
    "role": "user"
    "content": "define both"
  },
  {
    "role": "assistant"
    "content": "definition of abc and def"
  },
]

As stated, we glue prompt + few shots on start and add current user query in the end (with the prefix). After that prefix is not present anymore in 'real' query messages history.

Thanks so much for sharing your approach!
I'm going to CC @srbalakr from the ACS team who worked most recently on the query generation for their thoughts. I think PRs are always great to share with the community, even those that don't get merged, but if this produces overall better response quality across many queries/knowledge bases, then we may want it in main.

Yes please share the PR, I have also put up a PR to stabilize the generation for lengthy chats using function calls. It should address most of the concerns.

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

Re-opened, I'm still interested in this. I don't have multi-turn evaluation setup yet, only single-turn (as you can see in #967) so I haven't been able to evaluate this change programmatically.

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.