Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

split out CFDatetimeCoder, deprecate use_cftime as kwarg #9901

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1096,6 +1096,17 @@ DataTree methods
.. Missing:
.. ``open_mfdatatree``

Encoding/Decoding
=================

Coder objects
-------------

.. autosummary::
:toctree: generated/

coders.CFDatetimeCoder

Coordinates objects
===================

Expand Down
6 changes: 6 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ New Features
- Add ``unit`` - keyword argument to :py:func:`date_range` and ``microsecond`` parsing to
iso8601-parser (:pull:`9885`).
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_.
- Split out ``CFDatetimeCoder`` as public API in ``xr.coders``, make ``decode_times`` keyword argument
consume ``CFDatetimeCoder``.


Breaking changes
~~~~~~~~~~~~~~~~
Expand All @@ -50,6 +53,9 @@ Deprecations
- Finalize deprecation of ``closed`` parameters of :py:func:`cftime_range` and
:py:func:`date_range` (:pull:`9882`).
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_.
- Time decoding related kwarg ``use_cftime`` is deprecated. Use keyword argument
kmuehlbauer marked this conversation as resolved.
Show resolved Hide resolved
``decode_times=CFDatetimeCoder(use_cftime=True)`` in :py:func:`~xarray.open_dataset`, :py:func:`~xarray.open_dataarray`, :py:func:`~xarray.open_datatree`, :py:func:`~xarray.open_groups`, :py:func:`~xarray.open_zarr` and :py:func:`~xarray.decode_cf` instead.
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_.

Bug fixes
~~~~~~~~~
Expand Down
3 changes: 2 additions & 1 deletion xarray/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from importlib.metadata import version as _version

from xarray import groupers, testing, tutorial, ufuncs
from xarray import coders, groupers, testing, tutorial, ufuncs
from xarray.backends.api import (
load_dataarray,
load_dataset,
Expand Down Expand Up @@ -66,6 +66,7 @@
# `mypy --strict` running in projects that import xarray.
__all__ = ( # noqa: RUF022
# Sub-packages
"coders",
"groupers",
"testing",
"tutorial",
Expand Down
56 changes: 44 additions & 12 deletions xarray/backends/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
_normalize_path,
)
from xarray.backends.locks import _get_scheduler
from xarray.coders import CFDatetimeCoder
from xarray.core import indexing
from xarray.core.combine import (
_infer_concat_order_from_positions,
Expand Down Expand Up @@ -481,7 +482,10 @@ def open_dataset(
cache: bool | None = None,
decode_cf: bool | None = None,
mask_and_scale: bool | Mapping[str, bool] | None = None,
decode_times: bool | Mapping[str, bool] | None = None,
decode_times: bool
| CFDatetimeCoder
| Mapping[str, bool | CFDatetimeCoder]
| None = None,
decode_timedelta: bool | Mapping[str, bool] | None = None,
use_cftime: bool | Mapping[str, bool] | None = None,
concat_characters: bool | Mapping[str, bool] | None = None,
Expand Down Expand Up @@ -543,9 +547,10 @@ def open_dataset(
be replaced by NA. Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.
decode_times : bool or dict-like, optional
decode_times : bool, CFDatetimeCoder or dict-like, optional
If True, decode times encoded in the standard NetCDF datetime format
into datetime objects. Otherwise, leave them encoded as numbers.
into datetime objects. Otherwise, use ``CFDatetimeCoder`` or leave them
encoded as numbers.
Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.
Expand All @@ -569,6 +574,10 @@ def open_dataset(
raise an error. Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.

.. deprecated:: 2024.12.0
Please initialize it with ``CFDatetimeCoder`` and ``decode_times`` kwarg.

concat_characters : bool or dict-like, optional
If True, concatenate along the last dimension of character arrays to
form string arrays. Dimensions will only be concatenated over (and
Expand Down Expand Up @@ -698,7 +707,10 @@ def open_dataarray(
cache: bool | None = None,
decode_cf: bool | None = None,
mask_and_scale: bool | None = None,
decode_times: bool | None = None,
decode_times: bool
| CFDatetimeCoder
| Mapping[str, bool | CFDatetimeCoder]
| None = None,
decode_timedelta: bool | None = None,
use_cftime: bool | None = None,
concat_characters: bool | None = None,
Expand Down Expand Up @@ -761,9 +773,11 @@ def open_dataarray(
`missing_value` attribute contains multiple values a warning will be
issued and all array values matching one of the multiple values will
be replaced by NA. This keyword may not be supported by all the backends.
decode_times : bool, optional
decode_times : bool, CFDatetimeCoder or dict-like, optional
If True, decode times encoded in the standard NetCDF datetime format
into datetime objects. Otherwise, leave them encoded as numbers.
into datetime objects. Otherwise, use CFDatetimeCoder or leave them encoded as numbers.
Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.
decode_timedelta : bool, optional
If True, decode variables and coordinates with time units in
Expand All @@ -781,6 +795,10 @@ def open_dataarray(
represented using ``np.datetime64[ns]`` objects. If False, always
decode times to ``np.datetime64[ns]`` objects; if this is not possible
raise an error. This keyword may not be supported by all the backends.

.. deprecated:: 2024.12.0
Please initialize it with ``CFDatetimeCoder`` and ``decode_times`` kwarg.

concat_characters : bool, optional
If True, concatenate along the last dimension of character arrays to
form string arrays. Dimensions will only be concatenated over (and
Expand Down Expand Up @@ -903,7 +921,10 @@ def open_datatree(
cache: bool | None = None,
decode_cf: bool | None = None,
mask_and_scale: bool | Mapping[str, bool] | None = None,
decode_times: bool | Mapping[str, bool] | None = None,
decode_times: bool
| CFDatetimeCoder
| Mapping[str, bool | CFDatetimeCoder]
| None = None,
decode_timedelta: bool | Mapping[str, bool] | None = None,
use_cftime: bool | Mapping[str, bool] | None = None,
concat_characters: bool | Mapping[str, bool] | None = None,
Expand Down Expand Up @@ -961,9 +982,9 @@ def open_datatree(
be replaced by NA. Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.
decode_times : bool or dict-like, optional
decode_times : bool, CFDatetimeCoder or dict-like, optional
If True, decode times encoded in the standard NetCDF datetime format
into datetime objects. Otherwise, leave them encoded as numbers.
into datetime objects. Otherwise, use CFDatetimeCoder or leave them encoded as numbers.
Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.
Expand All @@ -987,6 +1008,10 @@ def open_datatree(
raise an error. Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.

.. deprecated:: 2024.12.0
Please initialize it with ``CFDatetimeCoder`` and ``decode_times`` kwarg.

concat_characters : bool or dict-like, optional
If True, concatenate along the last dimension of character arrays to
form string arrays. Dimensions will only be concatenated over (and
Expand Down Expand Up @@ -1118,7 +1143,10 @@ def open_groups(
cache: bool | None = None,
decode_cf: bool | None = None,
mask_and_scale: bool | Mapping[str, bool] | None = None,
decode_times: bool | Mapping[str, bool] | None = None,
decode_times: bool
| CFDatetimeCoder
| Mapping[str, bool | CFDatetimeCoder]
| None = None,
decode_timedelta: bool | Mapping[str, bool] | None = None,
use_cftime: bool | Mapping[str, bool] | None = None,
concat_characters: bool | Mapping[str, bool] | None = None,
Expand Down Expand Up @@ -1180,9 +1208,9 @@ def open_groups(
be replaced by NA. Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.
decode_times : bool or dict-like, optional
decode_times : bool, CFDatetimeCoder or dict-like, optional
If True, decode times encoded in the standard NetCDF datetime format
into datetime objects. Otherwise, leave them encoded as numbers.
into datetime objects. Otherwise, use CFDatetimeCoder or leave them encoded as numbers.
Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.
Expand All @@ -1206,6 +1234,10 @@ def open_groups(
raise an error. Pass a mapping, e.g. ``{"my_variable": False}``,
to toggle this feature per-variable individually.
This keyword may not be supported by all the backends.

.. deprecated:: 2024.12.0
Please initialize it with ``CFDatetimeCoder`` and ``decode_times`` kwarg.

concat_characters : bool or dict-like, optional
If True, concatenate along the last dimension of character arrays to
form string arrays. Dimensions will only be concatenated over (and
Expand Down
10 changes: 10 additions & 0 deletions xarray/coders.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
"""
This module provides coder objects that encapsulate the
"encoding/decoding" process.
"""

from xarray.coding.times import CFDatetimeCoder

__all__ = [
"CFDatetimeCoder",
]
21 changes: 17 additions & 4 deletions xarray/coding/times.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,11 @@
except ImportError:
cftime = None

from xarray.core.types import CFCalendar, NPDatetimeUnitOptions, T_DuckArray
from xarray.core.types import (
CFCalendar,
NPDatetimeUnitOptions,
T_DuckArray,
)

T_Name = Union[Hashable, None]

Expand Down Expand Up @@ -204,7 +208,10 @@ def _unpack_time_units_and_ref_date(units: str) -> tuple[str, pd.Timestamp]:


def _decode_cf_datetime_dtype(
data, units: str, calendar: str | None, use_cftime: bool | None
data,
units: str,
calendar: str | None,
use_cftime: bool | None,
) -> np.dtype:
# Verify that at least the first and last date can be decoded
# successfully. Otherwise, tracebacks end up swallowed by
Expand Down Expand Up @@ -311,7 +318,10 @@ def _decode_datetime_with_pandas(


def decode_cf_datetime(
num_dates, units: str, calendar: str | None = None, use_cftime: bool | None = None
num_dates,
units: str,
calendar: str | None = None,
use_cftime: bool | None = None,
) -> np.ndarray:
"""Given an array of numeric dates in netCDF format, convert it into a
numpy array of date time objects.
Expand Down Expand Up @@ -974,7 +984,10 @@ def _lazily_encode_cf_timedelta(


class CFDatetimeCoder(VariableCoder):
def __init__(self, use_cftime: bool | None = None) -> None:
def __init__(
self,
use_cftime: bool | None = None,
) -> None:
self.use_cftime = use_cftime

def encode(self, variable: Variable, name: T_Name = None) -> Variable:
Expand Down
Loading
Loading