GCPy 1.5.0

Release Date: 29 May 2024

Known issues

The pip install geoschem-gcpy command (as described on ReadTheDocs) is broken due to the xesmf 0.5.1 package having been de-listed from the Python Package Index (PyPi). For the time being, users should follow the manual installation instructions. See issue #326 for more information.

From CHANGELOG.md

Added

• Script gcpy/benchmark/modules/benchmark_utils.py, with common benchmark utility functions
• Script gcpy/benchmark/modules/benchmark_drydep.py, with code to create drydep velocity plots
• YAML tag plot_drydep in gcpy/benchmark/config/*.yml files
• Badge for build-gcpy-environment GitHub Action in README.md
• Badges in docs/source/index.rst
• GitHub action to push GCPy releases to PyPi
• Script ./release/changeVersionNumbers.sh, used to update version numbers in various files before release
• Mamba/Conda enviroment file docs/environment_files/read_the_docs_environment.yml, for building ReadTheDocs documentation
• Environment files docs/environment_files/gcpy_requirements.txt and docs/environment_files/read_the_docs_requirements.txt
• New benchmark script gcpy/benchmark/modules/benchmark_models_vs_sondes.py
• Added fixed level budget diagnostic to budget operations table
• Function rename_speciesconc_to_speciesconcvv in gcpy/benchmark/modules/benchmark_utils.py
• Function copy_file_to_dir in gcpy/util.py. This is a wrapper for shutil.copyfile.
• GitHub Action config file .github/workflows/stale.yml, which replaces StaleBot
• Added gcpy/community/format_hemco_data.py from @hannahnesser
• Added gcpy/examples/hemco/format_hemco_demo.py from @hannahnesser
• Added HCl to gcpy/benchmark/modules/emission_species.yml and GTChlorine to gcpy/benchmark/modules/emission_inventories.yml for GEOS-Chem 14.4.0
• Example script gcpy/examples/working_with_files/make_mask_file.py
• Convenience function replace_whitespace in gcpy/util.py
• Benchmark script gcpy/benchmark/modules/benchmark_scrape_gcclassic_timers.py
• Benchmark script gcpy/benchmark/modules/benchmark_scrape_gchp_timers.py
• 1-year benchmark scripts now produce GCC vs GCC and GCHP vs GCHP timing tables
• Functions gcc_vs_gcc_dirs, gchp_vs_gcc_dirs, gchp_vs_gchp_dirs, and get_log_filepaths in gcpy/benchmark/modules/benchmark_utils.py
• Script gcpy/benchmark/modules/benchmark_mass_cons_table.py, with code to create mass conservation tables
• Expanded statistics output in benchmark mass conservation tables
• Function get_datetimes_from_filenames in gcpy/benchmark/modules/benchmark_utils.py
• Function replace_whitespace in gcpy/util.py

Changed

• Bump pip from 23.2.1 to 23.3 (dependabot suggested this)
• Bump pypdf from 3.16.1 to 3.17.0 (dependabot suggested this)
• YAML tag operations_budget is now ops_budget_table in gcpy/benchmark/config/1yr_tt_benchmark.yml
• Now require matplotlib=3.8.0 in docs/environment_files/environment.yml (with other pegged versions)
• Now run the stale GitHub action at 00:00 UTC on the 1st of each month
• Renamed docs/environment_files/environment.yml to gcpy_environment.yml
• environment.yml links to docs/environment_files/gcpy_environment.yml
• requirements.txt links to docs/environment_files/requirements.txt
• Python packages for RTD documenation builds from docs/environment_files/environment.yml
• Script benchmark_model_vs_obs.py now uses grid inquiry functions from grid.py to return data nearest to a (lat,lon) location
• Moved routine get_geoschem_level_metadata to gcpy/benchmark/modules/benchmark_utils.py
• Refactored get_vert_grid.py (in gcpy/grid.py) to accept the p_sfc argument; Also never-nested the if-block logic.
• benchmark_slurm.sh script now saves output to a log file with the same base name as the YAML config file
• benchmark_models_vs_obs.py now reads the observational data paths and metadata from 1yr_fullchem_benchmark.yml
• Moved benchmark code from gcpy/ to gcpy/benchmark/modules/
• Benchmark functions now call rename_speciesconc_to_speciesconcvv
• Create radionuclide, STE flux, and mass conservation tables for Ref and Dev versions in TransportTracers benchmarks
• Use new function copy_file_to_dir to copy the benchmark script and configuration file to the benchmark results folders
• Updated GitHub stalebot config file stale.yml with new issue/PR labels that should not go stale
• Updated benchmark driver scripts and config files to print GCClassic & GCHP timing information
• Updated documentation for GCPy 1.5.0 release
• Converted Github issue templates to issue forms using YAML definition files

Fixed

• CS inquiry functions in gcpy/cstools.py now work properly for xr.Dataset and xr.DataArray objects
• Prevent an import error by using seaborn-v0_8-darkgrid ingcpy/benchmark/modules/benchmark_models_vs_obs.py
• gcpy/file_regrid.py now creates GCHP files with DELP_DRY instead of DELPDRY
• Fixed bugs in GCHP vs GCHP sections of 1-yr benchmark run scripts to allow comparison of GCHP runs with different grid resolutions
• Fixed silent bug in transport tracer benchmark GCC vs GCHP mass
  tables preventi ng them from being generated
• Import error in gcpy/examples/diagnostics/compare_diags.py
• Added missing n_cores to gcpy/examples/diagnostics/compare_diags.yml
• Added missing plot_drydep option to gcpy/gcpy/benchmark/config/1yr_ch4_benchmark.yml
• Add docs/requirements.txt symbolic link to docs/environment_files/read_the_docs_requirements.txt for RTD builds
• gcpy/file_regrid.py now tests if lon_bnds, lat_bnds are in the dataset before trying to drop them

Removed

• Example script gcpy/examples/plotting/mda8_o3_timeseries.py
• Removed Pylint GitHub action
• Environment file docs/environment_files/environment.yml
• Environment file docs/environment_files/requirements.txt
• Removed awscli from the GCPy environment; version 2 is no longer available on conda-forge or PyPi
• GitHub config files .github/stale.yml and .github/no-response.yml
• Routine make_benchmark_mass_conservation_table in benchmark_funcs.py; this is now obsolete

Pull Requests Included

• Bump pypdf from 3.16.1 to 3.17.0 by @dependabot in #293
• Add option to create drydep velocity plots from benchmark output by @yantosca in #280
• Add a GitHub action to auto-push GCPy releases directly to PyPi by @yantosca in #294
• file_regrid.py now uses DELP_DRY for delta-dry pressure variable name when creating GCHP files by @yantosca in #297
• Fix bugs in 1-yr benchmarks for different GCHP grid resolutions by @lizziel in #298
• Fix bugs in GCC vs GCHP transport tracer benchmark mass tables by @lizziel in #300
• Implement several minor bug fixes (matplotlib 3.8+, GitHub actions, remove obsolete features) by @yantosca in #283
• Split python packages needed for RTD documentation builds out of the GCPy conda environment and into a separate environment; remove awscli by @yantosca in #295
• Bug fix for operations budget name in transport tracer benchmark run script by @lizziel in #301
• Add inquiry functions to return nearest model data to a (lat,lon) location; Update benchmark model vs. obs scripts; Add models vs sondes output by @yantosca in #277
• Add fixed level diagnostic to operations budget table by @lizziel in #304
• Added missing options to 1yr_ch4_benchmark.yml config file by @msulprizio in #307
• Move remaining benchmark-related code out of gcpy and into gcpy/benchmark/modules folder (plus other improvements) by @yantosca in #305
• Replace stalebot with "stale" GitHub action by @yantosca in #315
• Fix issues in cubed-sphere regridding by @yuanjianz in #311
• Add example script to create mask files by @yantosca in #317
• Add HCl to emissions_species.yml for GEOS-Chem 14.4.0 by @msulprizio in #320
• Add benchmark scripts to scrape timing information from log files by @yantosca in #319
• Update mass conservation tables to include Ref & Dev in the same file by @yantosca in #309
• Bump requests from 2.31.0 to 2.32.0 by @dependabot in #321
• Update Github issue templates to use issue forms by @msulprizio in #323
• Bump requests from 2.31.0 to 2.32.2 in /docs/environment_files by @dependabot in #324

New Contributors

• @yuanjianz made their first contribution in #311

Full Changelog: 1.4.2...1.5.0

GCPy 1.4.2

Release date 26 Jan 2024

Pull Requests Included

• Add GitHub action to build GCPy environment and test installation by @yantosca in #286
• Fix dictionary key error when benchmark plots are generated sequentially by @yantosca in #287
• Use pegged version numbers for the GCPy environment; Also add testing environment without pegged versions; Updated GitHub actions by @yantosca in #289

Full Changelog: 1.4.1...1.4.2

GCPy 1.4.1

Release date 08 Dec 2023

From CHANGELOG.md

Fixed

• Now use the proper default value for the --weightsdir argument to gcpy/file_regrid.py

Pull Requests Included

• Bug fix: Use proper default value for weightsdir in file_regrid.py by @yantosca in #279

Full Changelog: 1.4.0...1.4.1

GCPy 1.4.0

Release date 2023-11-20

From CHANGELOG.md

Added

• Added C2H2 and C2H4 to emission_species.yml
• Updated species_database.yml for consistency with GEOS-Chem 14.2.0
• Added .github/ISSUE_TEMPLATE/config.yml file w/ Github issue options
• Added CONTRIBUTING.md and SUPPORT.md, replacing docs/source/Contributing.rst and docs/source/Report_Request.rst
• Added option to pass the benchmark type to plotting routines
• Updated AUTHORS.txt as of Apr 2023 (concurrent w/ GEOS-Chem 14.2.0)
• Added ReadTheDocs badge in README.md
• Added .readthedocs.yaml to configure ReadTheDocs builds
• Added cloud benchmarking YAML configuration files to benchmark/cloud folder
• Added README.md files in gcpy/benchmark directory structure
• Added benchmark/modules/benchmark_models_vs_obs.py script
• Added benchmark/modules/GC_72_vertical_levels.csv file
• Added multi_index_lat keyword to reshape_MAPL_CS function in gcpy/util.py
• Added FURA to emission_species.yml and benchmark_categories.yml
• Added new routine format_number_for_table in gcpy/util.py
• Added module gcpy/cstools.py with utility functions for cubed-sphere grids
• Added new routine verify_variable_type function in gcpy/util.py
• Added new routine format_number_for_table in util.py
• Added BrSALA and BrSALC to emission_species.yml
• Added options:n_cores to all benchmark YAML config files
• Added __init__.py files in subfolders of gcpy/gcpy
• gcpy/benchmark/modules/*.py scripts are now chmod 644
• Added ENCODING = "UTF-8" to gcpy/constants.py
• Added statement from dask.array import Array as DaskArray in gcpy plot.py
• Added SLURM run script gcpy/benchmark/benchmark_slurm.sh
• Added gcpy/plot/gcpy_plot_style style sheet for title and label default settings
• Added gcpy/gcpy_plot_style style sheet for title and label default settings
• Added new cubed-sphere grid inquiry functions to gcpy/cstools.py
• Added functions get_ilev_coord and get_lev_coord to gcpy/grid.py
• Add tk package to docs/environment_files/environment.yml

Changed

• Simplified the Github issues templates into two options: new-feature-or-discussion.md and question-issue.md
• The GitHub PR template is now named ./github/PULL_REQUEST_TEMPLATE.md
• Updated badge links in README.md
• Construct ops budget table filename without using the label argument
• Updated species_database.yml for consistency with GEOS-Chem 14.2.0
• Renamed TransportTracers species in benchmark_categories.yml, run_1yr_tt_benchmark.py, and in documentation
• YAML files in benchmark/ have been moved to benchmark/config
• Models vs. O3 obs plots are now arranged by site latitude from north to south
• Routine print_totals now prints small and/or large numbers in scientific notation
• Truncate names in benchmark & emissions tables to improve readability
• Add TransportTracers species names to gcpy/emissions_*.yml files
• Now pass n_job=config["options"]["n_cores"] to benchmark plotting routines
• Script benchmark.py to benchmark_funcs.py to remove a name collision
• Folder gcpy/benchmark is now gcpy/gcpy/benchmark
• Folder benchmark/modules is now gcpy/gcpy/benchmark/modules
• Folder gcpy/examples is now gcpy/gcpy/examples
• Pass sys.argv to the main() routine of run_benchmark.py, compare_diags.py
• Updated docs/environment_files/environment.yml for MambaForge (also added gridspec)
• Now use pypdf instead of PyPDF2 in plot.py and util.py
• Added coding suggestions made by pylint where possible
• Abstracted and never-nested code from six_plot into functions (in plot.py)
• Added main() routine to gcpy/file_regrid.py; Also added updates suggested by Pylint
• Fixed broken regridding code in gcpy/file_regrid.py; also refactored for clarity
• Rewrote Regridding.rst page; Confirmed that regridding examples work properly
• Now allow plot_val to be of type dask.array.Array in plot.py routines six_plot and single_panel
• Now add if statements to turn of Parallel() commands when n_jobs==1.
• Do not hardwire fontsize in gcpy/plot.py; get defaults from gcpy_plot_style
• gcpy/plot.py has been split up into smaller modules in the gcpy/plot folder
• Updated and cleaned up code in gcpy/regrid.py
• Example scriptsplot_single_level and plot_comparisons can now accept command-line arguments
• Example scripts plot_single_level.py, plot_comparisons.py, compare_diags.py now handle GCHP restart files properly
• Now specify the X11 backend with by setting the MPLBACKEND environment variable

Fixed

• Generalized test for GCHP or GCClassic restart file in regrid_restart_file.py
• Fixed bug in transport tracer benchmark mass conservation table file write
• Routine create_display_name now splits on only the first _ in species & diag names
• Prevent plot panels from overlapping in six-panel plots
• Prevent colorbar tick labels from overlapping in dynamic-range ratio plots
• Updated seaborn plot style names to conform to the latest matplotlib
• Set lev:positive and/or ilev:positive properly in regrid_restart_file.py and file_regrid.py
• Prevent overwriting of lev coord in file_regrid.py at netCDF write time
• Fixed bug in option to allow different units when making comparison plots

Removed

• Removed gchp_is_pre_13_1 arguments & code from benchmarking routines
• Removed is_pre_13_1 tags from *_benchmark.yml config files
• Removed benchmark_emission_totals.ipynb, this is obsolete
• Replaced gcpy/benchmark/README with README.md
• Removed gcpy_test_dir option from examples/diagnostics/compare_diags.*
• Removed docs/environment_files/gchp_regridding.yml environment file
• Removed gcpy/gcpy/benchmark/plot_driver.sh
• Made benchmark configuration files consistent

Pull Requests Included

• Add C2H2 and C2H4 to emissions_species.yml by @msulprizio in #208
• Simplify Github issue and pull request templates by @msulprizio in #209
• Update run_benchmark.py to allow for other benchmark types by @msulprizio in #212
• Allow for other benchmark types for GCHP plotting as well by @msulprizio in #213
• Bump requests from 2.26.0 to 2.31.0 by @dependabot in #221
• Optional mass accumulation table in 1-month benchmark by @lizziel in #216
• Feature/benchmark cloud plotting templates by @laestrada in #223
• Bug fixes in transport tracer benchmark code by @lizziel in #224
• Remove references to gcpy_test and gchp_is_pre_13_1 from benchmarking routines and config files by @yantosca in #222
• Remove gcpy_test YAML key from benchmark config files by @yantosca in #226
• Shorten the filename of the benchmark operations budget table by @yantosca in #227
• Update species database and benchmark categories for TransportTracer changes in GEOS-Chem 14.2.0 by @msulprizio in #225
• Add capability to plot GEOS-Chem Classic and/or GCHP data vs. O3 model observations by @yantosca in #234
• Add lumped furans to emission_species.yml and benchmark_categories.yml by @msulprizio in #235
• Arrange models vs. observations plots from north to south by site latitude by @yantosca in #236
• Fix formatting of mass & emission tables; add mass table option to 1-year TransportTracers benchmarks by @yantosca in #238
• GCGridObj tools for cubed-sphere grids by @lizziel in #239
• Fix issue where wrong GCHP grid boxes were being looked up in models vs. obs plots by @yantosca in #240
• Now use a consistent width for columns in emissions tables and mass table output by @yantosca in #248
• Update gcpy_env environment to install with MambaForge and to use latest package versions; Update scripts accordingly by @yantosca in #251
• Fixed broken regridding code in file_regrid.py; Updated GCPy regridding documentation by @yantosca in #253
• Peg xarray to version 2023.2.0 in the environment.yml file by @yantosca in #255
• Rename PassiveTracers to TransportTracers in run_1yr_tt_benchmark.py by @msulprizio in #256
• Update plot.py for more recent xarray; also allow dask arrays to be passed to single_panel, six_plot routines by @yantosca in #257
• Now allow specification of the number of cores for benchmark plotting jobs by @yantosca in #246
• Rearrange folders according to Python packaging best practices by @yantosca in #250
• Prevent subplots from overlapping in 6-panel plots; Prevent colorbar labels from overlapping by @yantosca in #260
• Make benchmark configuration files consistent by @msulprizio in #264
• Fixed online regridding code (file_regrid.py) for restart-to-restart file regridding only by @yantosca in #266
• Add GitHub actions: CodeQL, Pylint, Stale by @yantosca in https://github.com/geosche...

GCPy 1.3.3

Release date: 09 Mar 2023

Pull Requests Included

Added

• Add capability to generate CH4 benchmark output by @msulprizio in #186
• Added documentation for installing Conda 4.12.0 with Miniconda by @yantosca
• Add stretching parameters to GCHP regridding example by @kilicomu in #206

Changed

• Update SpeciesConc variable name to SpeciesConcVV for consistency with GEOS-Chem 14.1.0 updates by @msulprizio in #187
• Update benchmark table output to better show which diagnostics have Ref = Dev by @yantosca in #200
• Updated GCHP regridding documentation by @kilicomu in #205
• Additional cleanup of gcpy routines by @yantosca in #204

Fixed

• Fix bug in GCHP transport tracer budget table by @lizziel in #192
• Fixed GCHP regridding issues by @kilicomu in #190

Full Changelog: 1.3.2...1.3.3

GCPy 1.3.2

Release date: 26 Oct 2022

Changed

• Bug fix: Fixed malformed version declaration for cartopy (use == instead of =) in setup.py. This was preventing upload to conda-forge.

GCPy 1.3.1

Release date 26 Oct 2022

What Changed:

• Bug fix in setup.py, removed extraneous character
• Updated version numbers in various files

Full Changelog: 1.3.0...1.3.1

GCPy 1.3.0

Release date: 25 Oct 2022

What's New

• New features in benchmarking scripts (@lizziel, @yantosca)
  • Force garbage collection at end benchmarking functions (@yantosca)
  • Extra print statements (@lizziel)
  • Diff-of-diffs plots for 1-year benchmarks (@lizziel)
  • sparselt is now a GCPy requirement (@lizziel)
• Removed obsolete environment.yml files (@yantosca)
• Added requirements.yml to docs folder for Sphinx/RTD documentation (@yantosca)
• Added regrid_restart_file.py by @LiamBindle in #159
• Add fix for GEOS-Chem Classic restart file path by @msulprizio in #160

What's Changed

• Fixed several issues in benchmarking scripts (@laestrada, @lizziel, @yantosca), also see PR #164
  • Updates for new GCHP restart file format by @lizziel in #151
  • Fixed bug in budget_ox.py; The drydep loss of Ox for GCHP was 12x too high
  • Add OMP_NUM_THREADS and OMP_STACKSIZE in plot_driver.sh (@yantosca)
  • Increase requested memory to 50MB in plot_driver.sh (@yantosca)
  • Benchmark scripts print a message upon completion (@yantosca)
  • Linted several benchmarking routines with Pylint (@yantosca)
  • Rewrote algorithm of add_lumped_species_to_dataset for speed (@yantosca)
  • Can now specify the path to species_database.yml for 1yr benchmarks (@yantosca)
  • 1-yr benchmarks now save output in subdirs of the same path (@lizziel)
  • Avoid hardwiring restart file paths in benchmark scripts (@yantosca)
  • Now use outputs_subdir tag from YAML file for paths to diagnostic files (@yantosca)
  • Now use restarts_subdir tag from YAML file for paths to restart files (@yantosca)
  • GCPy now uses proper year for dev in 1-yr benchmarks (@laestrada)
  • Fixed date string issue in benchmarking scripts (@lizziel)
  • Updates for new GCHP restart file format (@lizziel)
• Updated environment.yml with package versions that work together (@yantosca)
• Updated the AUTHORS.txt and LICENSE.txt files (@yantosca)

Full Changelog: 1.2.0...1.3.0

GCPy: 1.2.0

Release date 02 May 2022

Added

• Added Parameter for single_panel to support return of all 6 cubedsphere plots
• Added flexible time period for benchmark plotting scripts

Changed

• Modified single_panel to vmin/vmax parameters with newer versions of matplotlib (>3.5.0)
• Modified run_benchmark script to select correct species database depending on benchmark type
• Modified filename for Ox budget
• Modified readthedocs build to use mamba instead of conda to fix build failures
• Modified benchmark plotting scripts to use a single run_benchmark.py script
• Modified benchmark categories and species database yaml files
• Fixed bug in mass conservation table percent difference

For installation instructions, please see our GCPy manual at https://gcpy.readthedocs.io.

GCPy: 1.1.0

Added

• Added date_time.py module to help manage datetime utility functions
• Added GLYC, HAC, and pFe to benchmark categories
• Added gcpy/budget_ox.py to compute Ox budgets from 1-yr benchmarks
• Added capability to use GCHP 13.1.0+ or legacy file names in benchmark scripts
• Added new methods dataset_reader and get_dataset_mean to util.py

Changed

• Modified benchmarking scripts to use yaml config files.
• Modified dry-run scripts to use yaml config files.
• Updated benchmark/run_1yr_fullchem_benchmark.py to call the budget_ox.py for GCC vs GCC benchmark generation.
  • NOTE: we are waiting to make sure that the GCHP benchmarks output wetdep fields before activating this feature for GCHP.
• Modified plotting methods in benchmark.py to compute the mean of datasets over the time dimension, if the "time_mean" keyword is passed.
  • This feature is used to generate annual mean plots from 1-yr benchmark output.
• Modified run_1yr_tt_benchmark.py and run_1yr_fullchem_benchmark.py to generate both annual mean and seasonal plots
• Fixed formatting and import order issues in benchmark.py, util.py, budget_ox.py, and the run_*benchmark.py scripts as identified by pylint.
• Modified budget_ox.py to use Ox instead of O3 for computing budget terms