Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Can't load 'area_t' from ACCESS-OM2 #117

Open
anton-seaice opened this issue Sep 22, 2023 · 3 comments
Open

[BUG] Can't load 'area_t' from ACCESS-OM2 #117

anton-seaice opened this issue Sep 22, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@anton-seaice
Copy link
Collaborator

Describe the bug

The access-nri-intake catalog doesn't open the tracer area field from ACCESS-OM2 results. If I try and load the 'area_t' variable from '01deg_jra55v140_iaf_cycle4_jra55v150_extension' I get a failed to load error.

To Reproduce

Run:
catalog['01deg_jra55v140_iaf_cycle4_jra55v150_extension'].search(variable='area_t').to_dask()

Actual Result:
Failed to load dataset with key='ocean_2d_area_t.fx'

Expected Result:
Dataset with x and y dimensions, and area_t variable.

Additional context

I suspect this is because there are multiple files with the area_t variable, but they are all the same (i.e. it's a field for information/analysis, not explicitly model output).

i.e., these files cannot be concatenated (with default settings) by xarray because they are all the same
['/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1000/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1001/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1002/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1003/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1004/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1005/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1006/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1007/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output1008/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output992/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output993/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output994/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output995/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output996/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output997/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output998/ocean/ocean-2d-area_t.nc', '/g/data/ik11/outputs/access-om2-01/01deg_jra55v140_iaf_cycle4_jra55v150_extension/output999/ocean/ocean-2d-area_t.nc']

@anton-seaice anton-seaice added the bug Something isn't working label Sep 22, 2023
@dougiesquire
Copy link
Collaborator

Thanks for reporting @anton-seaice. You're right that the issue is that the catalog is trying to combine multiple datasets, with no clear dimension coordinate to combine along.

I took a look at how this is dealt with in the cosima-cookbook, but it has the same issue, e.g. try:

import cosima_cookbook as cc

session = cc.database.create_session()

area_t = cc.querying.getvar(
    '01deg_jra55v140_iaf_cycle4_jra55v150_extension',
    'area_t',
    session,
)

The cc.querying.getvar function supports an argument (n) to restrict the total number of files, which I guess is how this is dealt with there. I'm not sure about the best way to deal with this off the top of my head in the catalog (if you have any suggestions?). In the meantime, you could restrict the files in a similar way to the cookbook with, e.g.:

cat['01deg_jra55v140_iaf_cycle4_jra55v150_extension'].search(
    path=".*output1000.*",
    variable='area_t'
).to_dask()

@marc-white
Copy link
Collaborator

I wonder if #192 is the same problem... will need to check when Gadi comes back to life.

@anton-seaice
Copy link
Collaborator Author

Yeah all the grid variables will have the same problem.

I think this is the same issue as intake/intake-esm#660

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants