-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory error when store_window=True
in pspec_run
#354
Comments
This is kind of an insurmountable issue if one wants to compute the windows func for all baselines and times. For many estimators however the wf doesn’t change along these axes, so in the H1C analysis for example we run the pipeline w store_window False, run our bl and time averaging, then go back and compute the wf once. I think leaving the current capability to compute and store all bltimes is needed for future estimators that may not have time abs bl independent wf, but if you want to take this problem on we could write a method that identifies time and blpaor independent estimators and just stores one or a few wfs in memory |
Okay, I see. I guess I was thinking about the second option you mention where you do not store the same array a thousand times but it might be difficult to make general enough. Also, do you think it needs to be float64 and not 32? |
Yeah I think it should be fairly straightforward to do this. Two things can be done to make it easier to store blpair and time-independent window funcs.
These changes might require updates to the |
Thanks for the suggestions! I'll keep it in the back of my head and do it eventually :) |
The option in
pspec_run
that allows to compute and store (approximate) window functions leads to a memory overload.For example, for two datasets being combined to give a
PSPec
object withuvp.Ntimes = 8052
,uvp.Nblpairs = 63
,uvp.Npols = 1
and one spectral window (164 frequency channels), the memory usage between a case with and without window functions is about 25Gb.Note:
uvp.window_function_array
is defined as an array offloat64
.First spotted in the validation notebook
hera-validation/test-series/0/test-0.2.0.ipynb
.The text was updated successfully, but these errors were encountered: