Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Benchmarks #1197

Open
10 of 19 tasks
guarin opened this issue May 10, 2023 · 2 comments
Open
10 of 19 tasks

New Benchmarks #1197

guarin opened this issue May 10, 2023 · 2 comments
Labels

Comments

@guarin
Copy link
Contributor

guarin commented May 10, 2023

We are in the process of updating our benchmark code and adding benchmarks on the ImageNet dataset. The goal of this issue is to coordinate and track progress of the update.

The new benchmark code lives in lightly/benchmarks and initial focus is on reference implementations for the various SSL methods following the settings from the respective papers. We'll limit training to 100 epochs and small batch sizes (2x128) for now due to limited compute resources.

Note: The benchmarking code is very much work in progress, there will be frequent and breaking changes.

Implemented Methods

Contribute

Contributions are very welcome! You can help us with the following:

  • Add one of the existing models to the new benchmarks. For this you can follow the SimCLR example.
  • Let us know if you spot any mistake in the models or benchmarking code.
  • Run benchmarks for different epochs / batch size.

Let us know if you are working on a model and we can help you out with reviews/compute :)

@guarin guarin pinned this issue May 10, 2023
@ibro45
Copy link
Contributor

ibro45 commented May 22, 2023

Will you be creating a tensorboard with training losses of all these networks? May be useful when a user wants to get an idea of how the loss behaves during training.

@guarin
Copy link
Contributor Author

guarin commented May 25, 2023

Hi, yes this is the goal :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

No branches or pull requests

2 participants