Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numba not supported for numpy masked array #3

Open
minghangli-uni opened this issue Apr 11, 2024 · 4 comments
Open

numba not supported for numpy masked array #3

minghangli-uni opened this issue Apr 11, 2024 · 4 comments

Comments

@minghangli-uni
Copy link

I am trying to generate iniitial conditions of temperature and salinity for 0.25deg with 75 vertical layers, related to issue136. However, I received an error related to numba,

Unsupported array type: numpy.ma.MaskedArray.

The error prevents the generation of initial fields.

@minghangli-uni
Copy link
Author

minghangli-uni commented Apr 12, 2024

Following steps in README , an error was encountered as follows,

File "/g/data/hh5/public/apps/miniconda3/envs/analysis3-23.10/lib/python3.10/site-packages/numba/core/typing/typeof.py", line 244, in _typeof_ndarray raise errors.NumbaTypeError(msg) numba.core.errors.NumbaTypeError: ^[[1mUnsupported array type: numpy.ma.MaskedArray.

The error arises from the numba library, specifically within the apply_weights function used for ESMF regridding. This library does not support numpy masked arrays, and this support has not been added yet. I am unsure how we previously generated these initial conditions with this script. After removing numba, I managed to generate the expected field, but with a noticeable decrease in speed.

A sanity check was conducted between the existing data and newly generated data without using numba, and they were found to be identical.

@minghangli-uni
Copy link
Author

For anyone interested in reproducing the error, consider the following simple snippet:

import numpy as np
import numba
data = np.ma.masked_array([1,10,15], mask=[0,1,0])

@numba.jit
def masked_array_mean(arr):
    return np.mean(arr)
print(masked_array_mean(data))

It returns the same error, NumbaTypeError: Unsupported array type: numpy.ma.MaskedArray.

@aekiss
Copy link
Contributor

aekiss commented Aug 16, 2024

Glad you found a workaround @minghangli-uni. Could you post a PR with your fix please? @ezhilsabareesh8 has hit the same problem and it would be nice to have a working version in the repo.

@minghangli-uni
Copy link
Author

Sure thing. Let me wrap it up and prepare a PR shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants