Skip to content

Issues: mlcommons/modelbench

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

102 Open 79 Closed
102 Open 79 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Daily Scheduled Test Failure bug Something isn't working
#473 opened Sep 21, 2024 by github-actions bot
Daily Scheduled Test Failure bug Something isn't working
#472 opened Sep 21, 2024 by github-actions bot
Tests don’t instantiate annotators
#469 opened Sep 20, 2024 by bkorycki
1
Daily Scheduled Test Failure bug Something isn't working
#468 opened Sep 20, 2024 by github-actions bot
Daily Scheduled Test Failure bug Something isn't working
#467 opened Sep 20, 2024 by github-actions bot
Register mistral-nemo-instruct-2407 SUT pointing to hf dedicated endpoint
#462 opened Sep 19, 2024 by bkorycki
1
Add modelgauge docs
#461 opened Sep 19, 2024 by bkorycki
Add latest modelgauge commit missed in merge
#460 opened Sep 19, 2024 by bkorycki
Unite hugging face secrets for consistency and to fix broken daily tests
#459 opened Sep 19, 2024 by rogthefrog
@rogthefrog
1
Investigate proxy for clients to provision a model
#458 opened Sep 19, 2024 by rogthefrog
4
run report includes code version
#451 opened Sep 16, 2024 by wpietri
Split v1 Benchmark by use case
#450 opened Sep 16, 2024 by wpietri
Split v1 Benchmark by language
#449 opened Sep 16, 2024 by wpietri
Automatically start/stop huggingface inference endpoints
#447 opened Sep 13, 2024 by bkorycki
New hazards with v1 prompts
#445 opened Sep 13, 2024 by bkorycki
Daily Scheduled Test Failure bug Something isn't working
#440 opened Sep 6, 2024 by github-actions bot
Add --benchmark option to job runner
#434 opened Sep 4, 2024 by bkorycki
Daily Scheduled Test Failure bug Something isn't working
#433 opened Sep 4, 2024 by github-actions bot
Daily Scheduled Test Failure bug Something isn't working
#430 opened Sep 3, 2024 by github-actions bot
Daily Scheduled Test Failure bug Something isn't working
#429 opened Sep 2, 2024 by github-actions bot
Daily Scheduled Test Failure bug Something isn't working
#428 opened Sep 1, 2024 by github-actions bot
Daily Scheduled Test Failure bug Something isn't working
#427 opened Aug 31, 2024 by github-actions bot
Daily Scheduled Test Failure bug Something isn't working
#425 opened Aug 30, 2024 by github-actions bot
show errors and/or missed items in the reports
#424 opened Aug 29, 2024 by wpietri
All logs are in json format with --json-logs
#423 opened Aug 29, 2024 by bkorycki
ProTip! Follow long discussions with comments:>50.