Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

global flag in data stage shoots up memory usage #49

Open
raehik opened this issue Jun 8, 2023 · 1 comment
Open

global flag in data stage shoots up memory usage #49

raehik opened this issue Jun 8, 2023 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@raehik
Copy link
Collaborator

raehik commented Jun 8, 2023

Setting --global_ 1 when running cmip26.py balloons memory usage. @arthurBarthe offers that an xarray op used there might have changed behaviour.

The readme uses --global_ 1 in its example MLflow-wrapped command. That should probably be changed or come with a warning. It's set to 0 by default in cmip26.py.

@raehik raehik self-assigned this Jun 8, 2023
@arthurBarthe
Copy link
Collaborator

Probably a minor issue at the moment. The global parameter is set to 1 so that when we use the full data, it is "periodized" along the longitudinal dimension prior to the data processing so that the convolution operations do not incur data loss along that dimension. Better in order to have nicer plots (without a missing vertical band).

@raehik raehik added the bug Something isn't working label Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants