Skip to content

Commit

Permalink
test(snapshots): use devtools snapshot fixtures (#2189)
Browse files Browse the repository at this point in the history
* import snapshot fixtures from modflow-devtools and remove them from conftest.py
* switch to dev version of devtools in environment.yml, use micromamba in rtd.yml ci
  • Loading branch information
wpbonelli authored May 15, 2024
1 parent acfd0d3 commit a7ee6b2
Show file tree
Hide file tree
Showing 4 changed files with 19 additions and 119 deletions.
26 changes: 15 additions & 11 deletions .github/workflows/rtd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:

defaults:
run:
shell: bash
shell: bash -l {0}

steps:
- name: Checkout flopy repo
Expand All @@ -32,18 +32,22 @@ jobs:
echo $GITHUB_REF
echo $GITHUB_EVENT_NAME
- name: Setup Python
uses: actions/setup-python@v5
- name: Setup Micromamba
uses: mamba-org/setup-micromamba@v1
with:
python-version: 3.9
cache: 'pip'
cache-dependency-path: pyproject.toml
environment-file: etc/environment.yml
cache-environment: true
cache-downloads: true
create-args: >-
python=3.12
init-shell: >-
bash
powershell
- name: Upgrade pip
run: python -m pip install --upgrade pip

- name: Install flopy and dependencies
run: pip install ".[test, doc, optional]"
- name: Install Python dependencies
run: |
pip install --upgrade pip
pip install .
- name: Workaround OpenGL issue on Linux
if: runner.os == 'Linux'
Expand Down
6 changes: 1 addition & 5 deletions DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -350,11 +350,7 @@ By default, `pytest-benchmark` will only print profiling results to `stdout`. If

### Snapshot testing

Snapshot testing is a form of regression testing in which a "snapshot" of the results of some computation is verified and captured by the developer to be compared against when tests are subsequently run. This is accomplished with [`syrupy`](https://github.com/tophat/syrupy), which provides a `snapshot` fixture overriding the equality operator to allow comparison with e.g. `snapshot == result`. A few custom fixtures for snapshots of NumPy arrays are provided in `autotest/conftest.py`:

- `array_snapshot`: saves an array in a binary file for compact storage, can be inspected programmatically with `np.load()`
- `text_array_snapshot`: flattens an array and stores it in a text file, compromise between readability and disk usage
- `readable_array_snapshot`: stores an array in a text file in its original shape, easy to inspect but largest on disk
Snapshot testing is a form of regression testing in which a "snapshot" of the results of some computation is verified and captured by the developer to be compared against when tests are subsequently run. This is accomplished with [`syrupy`](https://github.com/tophat/syrupy) via [fixtures defined in `modflow-devtools`](https://modflow-devtools.readthedocs.io/en/latest/md/snapshots.html).

By default, tests run in comparison mode. This means a newly written test using any of the snapshot fixtures will fail until a snapshot is created. Snapshots can be created/updated by running pytest with the `--snapshot-update` flag.

Expand Down
103 changes: 1 addition & 102 deletions autotest/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

# import modflow-devtools fixtures

pytest_plugins = ["modflow_devtools.fixtures"]
pytest_plugins = ["modflow_devtools.fixtures", "modflow_devtools.snapshots"]


# constants
Expand Down Expand Up @@ -163,104 +163,3 @@ def pytest_report_header(config):
f"{optional} packages not found: {', '.join(not_found)}"
)
return "\n".join(lines)


# snapshot classes and fixtures

from syrupy.extensions.single_file import (
SingleFileSnapshotExtension,
WriteMode,
)
from syrupy.types import (
PropertyFilter,
PropertyMatcher,
SerializableData,
SerializedData,
)


def _serialize_bytes(data):
buffer = BytesIO()
np.save(buffer, data)
return buffer.getvalue()


class BinaryArrayExtension(SingleFileSnapshotExtension):
"""
Binary snapshot of a NumPy array. Can be read back into NumPy with
.load(), preserving dtype and shape. This is the recommended array
snapshot approach if human-readability is not a necessity, as disk
space is minimized.
"""

_write_mode = WriteMode.BINARY
_file_extension = "npy"

def serialize(
self,
data,
*,
exclude=None,
include=None,
matcher=None,
):
return _serialize_bytes(data)


class TextArrayExtension(SingleFileSnapshotExtension):
"""
Text snapshot of a NumPy array. Flattens the array before writing.
Can be read back into NumPy with .loadtxt() assuming you know the
shape of the expected data and subsequently reshape it if needed.
"""

_write_mode = WriteMode.TEXT
_file_extension = "txt"

def serialize(
self,
data: "SerializableData",
*,
exclude: Optional["PropertyFilter"] = None,
include: Optional["PropertyFilter"] = None,
matcher: Optional["PropertyMatcher"] = None,
) -> "SerializedData":
buffer = StringIO()
np.savetxt(buffer, data.ravel())
return buffer.getvalue()


class ReadableArrayExtension(SingleFileSnapshotExtension):
"""
Human-readable snapshot of a NumPy array. Preserves array shape
at the expense of possible loss of precision (default 8 places)
and more difficulty loading into NumPy than TextArrayExtension.
"""

_write_mode = WriteMode.TEXT
_file_extension = "txt"

def serialize(
self,
data: "SerializableData",
*,
exclude: Optional["PropertyFilter"] = None,
include: Optional["PropertyFilter"] = None,
matcher: Optional["PropertyMatcher"] = None,
) -> "SerializedData":
return np.array2string(data, threshold=np.inf)


@pytest.fixture
def array_snapshot(snapshot):
return snapshot.use_extension(BinaryArrayExtension)


@pytest.fixture
def text_array_snapshot(snapshot):
return snapshot.use_extension(TextArrayExtension)


@pytest.fixture
def readable_array_snapshot(snapshot):
return snapshot.use_extension(ReadableArrayExtension)
3 changes: 2 additions & 1 deletion etc/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ dependencies:
- filelock
- jupyter
- jupytext
- modflow-devtools
- pip:
- git+https://github.com/MODFLOW-USGS/modflow-devtools.git
- pytest!=8.1.0
- pytest-benchmark
- pytest-cov
Expand Down

0 comments on commit a7ee6b2

Please sign in to comment.