Skip to content

Commit

Permalink
Merge branch 'fix-typos-reviews' into 'candidate-release'
Browse files Browse the repository at this point in the history
Fix typos found in reviews

See merge request water/computational-tools/surface-water-work/hyswap!87
  • Loading branch information
ehinman committed Jul 24, 2024
2 parents 2f57764 + 40540c1 commit c1b4636
Show file tree
Hide file tree
Showing 10 changed files with 42 additions and 51 deletions.
15 changes: 4 additions & 11 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,17 @@ cover/
*.mo
*.pot

# Django stuff:
# Django:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
# Flask:
instance/
.webassets-cache

# Scrapy stuff:
# Scrapy:
.scrapy

# Sphinx documentation
Expand Down Expand Up @@ -114,7 +114,7 @@ ipython_config.py
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
# Celery:
celerybeat-schedule
celerybeat.pid

Expand Down Expand Up @@ -154,13 +154,6 @@ dmypy.json
# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# MacOS
*.DS_Store

Expand Down
4 changes: 0 additions & 4 deletions docs/source/examples/cumulative_hydrograph_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,6 @@ In this example we will fetch 20 years of data from a single site and then use t
df, md = dataretrieval.nwis.get_dv(
'06803495', start='2001-01-01', end='2021-12-31')

.. plot::
:context:
:include-source:

# plot the cumulative streamflow hydrograph
fig, ax = plt.subplots(figsize=(8, 5))
ax = hyswap.plots.plot_cumulative_hydrograph(
Expand Down
2 changes: 1 addition & 1 deletion docs/source/examples/flow_duration_curve_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Flow Duration Curves
--------------------

These examples show how flow duration curves can be constructed by fetching data from NWIS using `dataretrieval`, analyzing that data using functions provided by `hyswap` (:obj:`hyswap.exceedance.calculate_exceedance_probability_from_values_multiple`), and then plotted using another `hyswap` function (:obj:`hyswap.plots.plot_flow_duration_curve`).
For more information on flow duration curves, see the USGS report titled "Flow-duration curves" by James K. Searcy and published in 1959 (https://doi.org/10.3133/wsp1542A).
For more information on flow duration curves, see the 1959 USGS report by James K. Searcy, titled `"Flow-duration curves" <https://doi.org/10.3133/wsp1542A>`_.


Creating a Flow Duration Curve for Site 01646500
Expand Down
2 changes: 1 addition & 1 deletion docs/source/examples/percentile_calculation_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Calculating Historic Percentiles for One Site
The :obj:`hyswap.percentiles.calculate_fixed_percentile_thresholds` function
is used to calculate a set of percentile thresholds given a set of data.
This function simply calculates one set of fixed percentile thresholds using all available data,
and is not intended to be used for calculating percentile thresholds separatly for individual days of the year.
and is not intended to be used for calculating percentile thresholds separately for individual days of the year.

By default this method calculates percentiles using the Weibull distribution
with an alpha parameter of 0 and a beta parameter of 0. The Weibull
Expand Down
2 changes: 1 addition & 1 deletion docs/source/examples/similarity_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Similarity Measures

Sometimes it is helpful to compare the relationships between a set of streamgaging stations and their respective measurements. These examples showcase the usage of the functions in the `similarity` module to quantify how similar streamflow records are across multiple streamgages. Matrices of similarity measures (e.g., correlations) are calculated and visualized by generating heatmap visualizations via the :obj:`hyswap.plots.plot_similarity_heatmap` function.

The `similarity` functions packaged in `hyswap` handle some of the data clean-up for you by ensuring the time-series of observations being compared acros the same dates, and by removing any missing data. This ensures that your results are not skewed by missing data or gaps in one of the time-series.
The `similarity` functions packaged in `hyswap` handle some of the data clean-up for you by ensuring the time-series observations are compared across the same dates, and by removing any missing data. This ensures that your results are not skewed by missing data or gaps in one of the time-series.


Pearson's *r* Correlations Between 5 Stations
Expand Down
34 changes: 19 additions & 15 deletions docs/source/meta/calculations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@ A quick-reference guide to the common types of calculations performed within the

Assumptions and Caveats
-----------------------
The ``hyswap`` package functions assume that provided streamflow data has been quality controlled. No checks on incorrect, missing, or negative values are performed. Users should perform any necessary QA/QC checks on the data prior to using ``hyswap`` functions. Additionally, ``hyswap`` does not detect artifacts or shifts in streamflow data that potentially violate statistical methods, such as:
- negative flows associated with tidal-influenced locations or other phenomena
- regulated flows or transitions from regulated to unregulated flows (or vice versa)
- major watershed changes
The ``hyswap`` package functions assume that provided streamflow data are quality controlled. No checks on incorrect, missing, or negative values are performed. Users should perform any necessary QA/QC checks on the data prior to using ``hyswap`` functions. Additionally, ``hyswap`` does not detect artifacts or shifts in streamflow data that potentially violate statistical methods, such as:

For users who are applying ``hyswap`` functions directly to data retrieved from USGS NWIS, please be aware that non-standard parameter names occasionally occur. Reasons for this could be to indicate that a site has an upstream and downstream sensor, a sensor was moved in the water column, or a site was measured multiple times. This package treats sites with non-standard parameter names as outliers and skips those sites. Users should take appropriate steps to check for non-standard parameter names and determine appropriate data handling steps based on their needs.
* negative flows associated with tidal-influenced locations or other phenomena
* regulated flows or transitions from regulated to unregulated flows (or vice versa)
* major watershed changes

For users who are applying ``hyswap`` functions directly to data retrieved from USGS NWIS, please be aware that non-standard parameter names occasionally occur (e.g. `00060_2_Mean` or `00060_incorrect_Mean` instead of simply `00060_Mean`). Reasons for this could be to indicate that a site has an upstream and downstream sensor, a sensor was moved in the water column, or a site was measured multiple times. This package treats sites with non-standard parameter names as outliers and skips those sites. Users should take appropriate steps to check for non-standard parameter names and determine appropriate data handling steps based on their needs.

Streamflow Percentiles
----------------------
Expand Down Expand Up @@ -58,7 +59,7 @@ Streamflow percentiles are a core calculation of ``hyswap`` that are used to det
| | the year. |
+---------------------------+-------------------------------------------+

By default, ``hyswap`` computes streamflow percentiles using the unbiased Weibull plotting position formula, :math:`i/(n+1)`, where *i* is the rank of an observation and *n* is the sample size (Weibull, 1939). The Weibull formula has been the standard approach used by hydrologists for generating flow-duration and flood-frequency curves `(Helsel and others, 2020)`_. Weibull plotting position does not set values to either 0 or 100, recognizing the existence of a non-zero probablity of exceeding the maximum or minimum observed value. For further discussion of plotting positions refer to `(Helsel and others, 2020)`_.
By default, ``hyswap`` computes streamflow percentiles using the unbiased Weibull plotting position expression, :math:`i/(n+1)`, where *i* is the rank of an observation and *n* is the sample size (Weibull, 1939). The Weibull formula has been the standard approach used by hydrologists for generating flow-duration and flood-frequency curves `(Helsel and others, 2020)`_. Weibull plotting position does not set values to either 0 or 100, recognizing the existence of a non-zero probablity of exceeding the maximum or minimum observed value. For further discussion of plotting positions refer to `(Helsel and others, 2020)`_.

``hyswap`` uses the ``numpy.percentile()`` implementation of the Weibull method (Type 6) for calculating percentiles. Additional methods of computing percentiles that exist in the ``numpy.percentile()`` function can be used in ``hyswap``. Users can refer to the `numpy function documentation <https://numpy.org/doc/stable/reference/generated/numpy.percentile.html>`_ for additional details.

Expand All @@ -75,42 +76,42 @@ Flow Categorization
^^^^^^^^^^^^^^^^^^^
Streamflow observations at a streamgage can be assigned a flow condition category using ``hyswap`` by use of the `hyswap` :obj:`hyswap.utils.categorize_flows` function. Streamflow percentiles or interpolated estimated streamflow percentiles are compared to a categorization schema. Multiple categorization schema are available in ``hyswap`` with the default being flow categories similar to those displayed on the USGS National Water Dashboard. Categorization schema are applicable to both variable and fixed percentile types. Available schema are described below:

* "NWD" -- Categorization schema similar to the USGS National Water Dashboard, *default*
"NWD" -- Categorization schema similar to the USGS National Water Dashboard, *default*
Categorizes streamflow across all range of possible streamflow magnitudes. Typically used with variable percentiles

.. image:: ../reference/nwd.png
:width: 800
:alt: Categorization schema of percentile ranges, labels, and color palette similar to the National Water Dashboard.

* "WaterWatch" -- Categorization schema similar to the USGS WaterWatch website
"WaterWatch" -- Categorization schema similar to the USGS WaterWatch website
Categorizes streamflow across all range of possible streamflow magnitudes. Typically used with variable percentiles

.. image:: ../reference/waterwatch.png
:width: 800
:alt: Categorization schema of percentile ranges, labels, and color palette similar to USGS WaterWatch.

* "NIDIS_Drought" -- Categorization schema similar to the NIDIS U.S. Drought Monitor
"NIDIS_Drought" -- Categorization schema similar to the NIDIS U.S. Drought Monitor
Categorizes streamflow across only low-flow conditions.

.. image:: ../reference/nidis_drought.png
:width: 800
:alt: Categorization schema of percentile ranges, labels, and color palette similar to NIDIS U.S. Drought Monitor.

* "WaterWatch_Drought" -- Categorization schema similar to the USGS WaterWatch Drought Conditions
"WaterWatch_Drought" -- Categorization schema similar to the USGS WaterWatch Drought Conditions
Categorizes streamflow across only low-flow conditions. Typically used with variable percentiles

.. image:: ../reference/waterwatch_drought.png
:width: 800
:alt: Categorization schema of percentile ranges, labels, and color palette similar to USGS WaterWatch Drought Conditions.

* "WaterWatch_Flood" -- Categorization schema similar to the USGS WaterWatch Flood Conditions
"WaterWatch_Flood" -- Categorization schema similar to the USGS WaterWatch Flood Conditions
Categorizes streamflow across only high-flow conditions. Typically used with fixed percentiles

.. image:: ../reference/waterwatch_flood.png
:width: 800
:alt: Categorization schema of percentile ranges, labels, and color palette similar to USGS WaterWatch Flood Conditions.

* "WaterWatch_BrownBlue" -- Categorization schema similar to the USGS WaterWatch categories but with an alternative color palette
"WaterWatch_BrownBlue" -- Categorization schema similar to the USGS WaterWatch categories but with an alternative color palette
Categorizes streamflow across all range of possible streamflow magnitudes. Typically used with variable percentiles

.. image:: ../reference/waterwatch_brownblue.png
Expand All @@ -132,11 +133,11 @@ The calculation of area-based runoff in ``hyswap`` involves the steps described
Workflow for Associating Streamgages with HUC8s for Area-Based Runoff Calculations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Spatial datasets describing the respective drainage basin boundaries of the streamgages and the boundaries of hydrologic cataloging units must be obtained that cover all areas of interest (e.g. CONUS). Geospatial boundaries of streamgages may be based on delineated gage drainage areas calculated using NHDPlus Version 1 data `(U.S. Geological Survey, 2011)`_ or determined via other watershed delineation approaches. HUC8 boundaries are contained within the `USGS Watershed Boundary Dataset (WBD) <https://www.usgs.gov/national-hydrography/watershed-boundary-dataset>`_.
Spatial datasets describing the respective drainage basin boundaries of the streamgages and the boundaries of hydrologic cataloging units must be obtained that cover all areas of interest (e.g. CONUS). Geospatial boundaries of streamgages may be based on delineated gage drainage areas calculated using NHDPlus Version 1 data `(U.S. Geological Survey, 2011)`_ or determined via other watershed delineation approaches. HUC8 boundaries are contained within the `USGS Watershed Boundary Dataset (WBD) <https://www.usgs.gov/national-hydrography/watershed-boundary-dataset>`_. Check out the `HyRiver <https://docs.hyriver.io/index.html>`_ software stack for APIs to access NHDPlus and WBD data products.

Each geospatial streamgage drainage basin boundary is overlain on a geospatial dataset of HUC8s (the polygons outlined in thick gray-brown lines in Figure 1 example) to determine the area of intersection within the two datasets. For each overlapping area of HUC8s and streamgage drainage basin boundaries, the fraction of the basin in the HUC8 and the fraction of the HUC8 in the basin are calculated. These fractions are then multiplied by each other to compute a weighting factor for each basin in the runoff calculation.

You can find an example intersection dataset between CONUS HUC8s and streamgage basins in the `hyswap-example-notebooks repository <https://code.usgs.gov/water/computational-tools/surface-water-work/hyswap-example-notebooks>`_. This intersection dataset was created using the `surface water geospatial data assembly repository <https://code.usgs.gov/water/computational-tools/surface-water-work/surface-water-geospatial-data-assembly>`_.
You can find an example intersection dataset between CONUS HUC8s and streamgage basins in the `example_data <https://github.com/DOI-USGS/hyswap/tree/main/example_notebooks/example_data>`_ folder within the `example_notebooks` folder of this repository. This intersection dataset was created using the `hyswap geospatial data assembly repository <https://code.usgs.gov/water/computational-tools/surface-water-work/hyswap-geospatial-data-assembly>`_.

Workflow for Estimating Area-Based Runoff
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -145,11 +146,14 @@ After obtaining a table of intersecting HUC8s and streamgage basins, the next st

1. For each unit of time with runoff data at one or more streamgages whose basins intersect the HUC8, obtain a weighted average of all runoff values.
2. If a HUC8 and a streamgage basin have near perfect overlap (e.g. the proportion of the HUC8's area in the basin is greater than 0.9 and the proportion of the basin's area in the HUC8 is greater than 0.9), simply use the runoff values from that streamgage basin. If perfect overlap does not exist, use a weighted average of runoff values from:

(a) all streamgage basins *contained* by the HUC8, and
(b) the smallest streamgage basin that *contains* the HUC8

Note that in cases where there is near perfect overlap for multiple streamgage basins and a HUC8, the streamgage basin-HUC8 intersection with the *highest* weight is used to estimate runoff. This is the same method used to determine the smallest streamgage basin that contains the HUC8: among all the streamgage basins that contain the HUC8 (in other words, the proportion of the HUC8's area in the streamgage basins is roughly 1), find the HUC8-streamgage basin intersection with the highest weight (which means the proportion of the streamgage basin's area in the HUC8 is the largest). See Figure 1 for an example of this workflow. Note that in Figure 1, 'Fraction#' is analogous to 'proportion of shape X's area in shape Y'.

An example Jupyter notebook exemplifying this workflow is available for `download <https://github.com/DOI-USGS/hyswap/blob/main/example_notebooks/regional_runoff_calculations.ipynb>`_ from the hyswap repository and can be viewed within the ``hyswap`` `documentation <https://doi-usgs.github.io/hyswap/examples/regional_runoff_calculations.html>`_.

.. image:: ../reference/huc8_runoff_example.gif
:width: 600
:alt: Map and table that provide an example of the computation of area-based runoff for a given HUC.
Expand Down Expand Up @@ -216,7 +220,7 @@ Ziel, F., 2021. The energy distance for ensemble and scenario reduction, Phil, T

.. _(Brakebill and others, 2011): https://doi.org/10.1111/j.1752-1688.2011.00578.x
.. _(Helsel and others, 2020): https://doi.org/10.3133/tm4A3
.. _(Jones and others, 2022): https://doi.org/10.3133/tm11A3
.. _(Jones and others, 2022): https://pubs.usgs.gov/tm/11/a3/
.. _(Magyar & Sambridge, 2023): https://doi.org/10.5194/hess-27-991-2023
.. _(U.S. Geological Survey, 2011): https://water.usgs.gov/lookup/getspatial?streamgagebasins
.. _(U.S. Geological Survey, 2023): http://dx.doi.org/10.5066/F7P55KJN
Expand Down
Loading

0 comments on commit c1b4636

Please sign in to comment.