Save all files in a task at the same time to avoid recomputing intermediate results

[bouweandela]

Description

Save all files in a task at the same time to avoid recomputing intermediate results.

This change is not backward compatible because it changes the return value of esmvalcore.preprocessor.save, which is part of the public API. Previously this function returned the filename, not it returns None on immediate saves and a dask.delayed.Delayed for delayed saves that can be requested with the compute=False argument.

Closes #2521
Closes #2042

Link to documentation: https://esmvaltool--2522.org.readthedocs.build/projects/ESMValCore/en/2522/api/esmvalcore.preprocessor.html#esmvalcore.preprocessor.save

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

Attention: Patch coverage is 87.50000% with 5 lines in your changes missing coverage. Please review.

Project coverage is 94.75%. Comparing base (21ce35a) to head (5b964c2).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
esmvalcore/preprocessor/__init__.py	87.09%	4 Missing ⚠️
esmvalcore/preprocessor/_io.py	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2522      +/-   ##
==========================================
- Coverage   94.77%   94.75%   -0.02%     
==========================================
  Files         249      249              
  Lines       14081    14106      +25     
==========================================
+ Hits        13345    13366      +21     
- Misses        736      740       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.