Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Streaming content for token usage #8902

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

TaoChenOSU
Copy link
Contributor

@TaoChenOSU TaoChenOSU commented Sep 18, 2024

Motivation and Context

OpenAI recently starts providing token usage information on their streaming chat completion API. This ADR opens the discussion on how we should consume that information within our StreamingChatMessageContent data structure.

Description

Contribution Checklist

@TaoChenOSU TaoChenOSU added the python Pull requests for the Python Semantic Kernel label Sep 18, 2024
@TaoChenOSU TaoChenOSU self-assigned this Sep 18, 2024
@github-actions github-actions bot changed the title Streaming content for token usage Python: Streaming content for token usage Sep 18, 2024
@markwallace-microsoft markwallace-microsoft added documentation and removed python Pull requests for the Python Semantic Kernel labels Sep 18, 2024
@TaoChenOSU TaoChenOSU linked an issue Sep 18, 2024 that may be closed by this pull request
@TaoChenOSU
Copy link
Contributor Author

Proposal 3 and 4 came from an offline discussion. They do include a refactoring of the existing content classes, which is not necessary for handling streaming usage information from OAI. However, I do like those two proposals as they further clean up our code base. Out of the two, I think proposal 4 is cleaner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Python: OpenAI streaming token usage
2 participants