Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Populate with CMIP6 like CVs #4

Merged
merged 3 commits into from
Aug 9, 2023

Conversation

matthew-mizielinski
Copy link
Contributor

Have brought in a set of CVs dictionaries for required fields -- process followed was

(First commit)

  • took "CMIP6Plus_CV.json" file produced when dividing CMIP6 CVs into generic_CVs for Prototype mip tables PCMDI/mip-cmor-tables#3, which included all fields that did not go into the generic_CV.json file
  • Split that file out into its components, one file per term using src/Deconstruction.ipynb notebook
    • note that the original CMIP6Plus_CV file is not in this repository
  • wrote src/construct_cmip6plus_cv.py to reconstruct the CMIP6Plus_CV.json file from the individual JSON files. Includes a list of all MD5 checksums (see note below)

(second commit)

  • The experiment id and source id information in the CV file embedded in PCMDI/cmip6-cmor-tables have cut down versions of the information in the WCRP-CMIP/CMIP6_CVs. Replaced the CMIP6Plus experiment_id and source_id files from the CMIP6_CV repository and added md5 checksums.
  • modified src/construct_cmip6plus_cv.py to correctly construct source_id and experiment_id entries in the CV JSON file.
  • Reconstructed the CMIP6Plus_CV.json file

The construction script should now be able to take any updated JSON files for the individual terms and reconstruct the part needed to drive CMOR.

In the src/ directory there is a cv_checksums module that contains functions for setting and validating the md5 checksums.

To be clear: The functions to handle MD5 checksums have been copied from other tools. The functions look at the dictionaries they are given, remove the original checksum entry, compute the checksum of the JSON string representing the dictionary and then adds this to the dictionary. As such these are a reasonably robust method of monitoring for manual changes, which is what I've used it for in the past elsewhere when I don't want a user manually editing carefully controlled information.

…hey have additional info compared to that in the CV file used by CMOR. Ammended construct_cmip6plus_cv.py script accordingly and added previous CV commit to checksums dictionary
@durack1 durack1 merged commit a995a8e into main Aug 9, 2023
@durack1 durack1 deleted the issue2_matthew-mizielinski_CMIP6like_CVs branch August 9, 2023 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants