CMORise CCI Soil Moisture Gapfilled product

[morobking]

Description

This is for ESA CMUG work package OWP4.3, updating and adding datasets for later scientific work packages.
This PR is to update the existing CMORiser for the ESA CCI Soil Moisture dataset to include the Gap Filled version of the product.
This data is documented and available from here

-Closes (partially) #4234

• Link to documentation:

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🛠 This pull request has a descriptive title
• 🛠 Code is written according to the code quality guidelines
• 🛠 Documentation is available
• 🛠 Tests run successfully
• 🛠 The list of authors is up to date
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 All checks below this pull request were successful

New or updated recipe/diagnostic

• 🧪 Recipe runs successfully
• 🧪 Recipe is well documented
• 🧪 Figure(s) and data look as expected from literature
• 🛠 Provenance information has been added

New or updated data reformatting script

• 🛠 Documentation is available
• 🛠 The dataset has been added to the CMOR check recipe
• 🛠 The dataset has been added to the shared data pools of DKRZ and Jasmin by the @ESMValGroup/OBS-maintainers team
• 🧪 Numbers and units of the data look physically meaningful

---

To help with the number of pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[morobking]

I have add a set of options in the cmor config yml for ESACCI SOILMOISTURE so the gapfilled or combined version of the dataset can be CMORised with the same yml and py files.
The python file checks if the gapfilled version is being used and load the Uncertainty as a cube, this is different behaviour to the combined product which needs to use ancillary variables.

[valeriupredoi]

hi @morobking would you please run pre-commit install && pre-commit run --all many thanks 🍺

[morobking]

Just pushed the modified files after running it; the items all said passed when running git commit

[valeriupredoi]

looks good to me, cheers very much @morobking 🍺 @axel-lauer when you have a minute, could you pls have a quick looksee and merge, pls mate 🍺

[axel-lauer]

Thanks @morobking for extending this CMORizer. I took a look and have a few comments.

• I think it would be great to support both dataset versions (COMBINED and COMBINED_GAPFILLED). At the moment, downloader and formatter seem to support only one version.
• I would find it helpful to add the possibility to automatically download the gapfilled data to the downloading script.
• A new version (v9.2) of the dataset is already available and could be used instead of v9.1.
• Even after manually downloading the gapfilled data, I did not succeed running the formatting script and end up with the following error message

OSError: [Errno -101] NetCDF: HDF error: '/work/bd0854/b380103/download/Tier2/ESACCI-SOILMOISTURE/ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED_GAPFILLED-19791007000000-fv09.2.nc'

Not sure if that is related to a borken file in v9.2 or something else. It would be great if this could be checked.

[axel-lauer]

Suggested change

	version 'L3S-SSMV-COMBINED_GAPFILLED-fv09.1'
	version: 'L3S-SSMV-COMBINED_GAPFILLED-fv09.1'

[morobking]

Thanks @axel-lauer

I've corrected the missing : in the yml file.

I get errors with 1979 for the V9.2 of the product - I'm testing other years now. V9.1 does not go back as far as 1979.

At the moment, I've written the yml file so you comment out the version of the dataset you do not want to CMORise. Is there a better way to do this? Would a separate yml file be better that still uses the same py file?

Downloader - that would be a extra version or modification to https://github.com/ESMValGroup/ESMValTool/blob/cmorise_cci_sm_gapfilled/esmvaltool/cmorizers/data/downloaders/datasets/esacci_soilmoisture.py
I think I've updated downloaders before but I've not used ESMValTool to download data as yet.