Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make api_key param optional in LLMEvaluator #8340

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Make api_key param optional in LLMEvaluator #8340

wants to merge 9 commits into from
+8 −4

Conversation

srini047
Copy link
Contributor

@srini047 srini047 commented Sep 7, 2024 •
edited
Loading

Related Issues

Proposed Changes:

After allowing the support to LocalLLM models through Evaluators (featured here) the api_key param needs to be made optional to support LocalLLMs in the LLMEvaluator which acts as a base class for FaithfulnessEvaluator and ContextRelevanceEvaluator.

How did you test it?

Run the unit tests on the entire evaluators.

Notes for the reviewer

Would like to know if this is fine. Fixing #8330 will help me enhance the PR.

Checklist

@srini047 srini047 requested a review from a team as a code owner September 7, 2024 08:11
@srini047 srini047 requested review from Amnah199 and removed request for a team September 7, 2024 08:11
@github-actions github-actions bot added topic:tests type:documentation Improvements on the docs labels Sep 7, 2024
@coveralls
Copy link
Collaborator

coveralls commented Sep 7, 2024 •
edited
Loading

Pull Request Test Coverage Report for Build 10948952322

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 5 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.003%) to 90.358%
Files with Coverage Reduction New Missed Lines %
components/evaluators/llm_evaluator.py 5 95.08%
Totals Coverage Status
Change from base Build 10945167196: 0.003%
Covered Lines: 7338
Relevant Lines: 8121
💛 - Coveralls

@srini047
Copy link
Contributor Author

@davidsbatista @Amnah199 Thanks for the suggestions. Committed those.

@CLAassistant
Copy link

CLAassistant commented Sep 16, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidsbatista davidsbatista davidsbatista left review comments

@Amnah199 Amnah199 Amnah199 approved these changes

Assignees
No one assigned
Labels
topic:tests type:documentation Improvements on the docs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

API key should be made optional in LLMEvaluator and co
5 participants
@srini047 @coveralls @CLAassistant @davidsbatista @Amnah199