Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding optima to tabular benchmarks (LCBench) #10

Closed
wants to merge 2 commits into from

Conversation

Neeratyoy
Copy link
Contributor

For tabular benchmarks, this is a must-have option.

Can also be added as a property to the Result object.

Currently, I want to access this as a benchmark property ideally and not have to query for this. Hence, adding it for each row.

Happy to discuss the best way to retrieve it.

This option can be kept general and continuous benchmarks can have this as None. Also could be used as a proxy check to test if a benchmark is tabular or surrogate.

@eddiebergman
Copy link
Contributor

Going to close this since it's superseed by the new metrics API, however I will add this in!

@eddiebergman
Copy link
Contributor

The new method will be:

# Only available on tabular datasets, uses entries in the table
benchmark.table_optimums["bal_accuracy"].error # for minimize optimum
benchmark.table_optimums["bal_accuracy"].score # for maximize optimum
# Available on all datasets, just takes the metrics definition
benchmark.metric_optimums["bal_accuracy"].error # for minimize optimum
benchmark.metric_optimums["bal_accuracy"].score # for maximize optimum

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants