在 xarray 中重新采样时可以扩展维度吗？
Possible to expand dimensions while resampling in xarray?
假设我有一个索引为 (x, y, time) 的数据集：
import xarray as xr, pandas as pd, numpy as np
x = np.linspace(-110, -90, 5)
y = np.linspace(23, 30, 5)
time = pd.date_range('1990-01-01', '2100-12-01', freq='MS')
da = xr.DataArray(
np.random.random(size=(5, 5, len(time))),
dims=['x', 'y', 'time'],
coords=[x, y, time],
name='temperature',
)
有没有办法执行任意重采样或 groupby 缩减，从而沿新维度（分组维度除外）扩展数组？类似于 np.quantile 或 xhistogram.xarray.histogram?
例如，np.quantile 在结果中创建了一个新维度，这会导致错误：
In [15]: da.resample(time='YS').reduce(np.quantile, q=[0.05, 0.5, 0.95])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [15], in <cell line: 1>()
----> 1 da.resample(time='YS').reduce(np.percentile, q=[0.05, 0.5, 0.95])
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:907, in DataArrayGroupByBase.reduce(self, func, dim, axis, keep_attrs, shortcut, **kwargs)
903 return ar.reduce(func, dim, axis, keep_attrs=keep_attrs, **kwargs)
905 check_reduce_dims(dim, self.dims)
--> 907 return self.map(reduce_array, shortcut=shortcut)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/resample.py:222, in DataArrayResample.map(self, func, shortcut, args, **kwargs)
179 """Apply a function to each array in the group and concatenate them
180 together into a new array.
181
(...)
218 The result of splitting, applying and combining this array.
219 """
220 # TODO: the argument order for Resample doesn't match that for its parent,
221 # GroupBy
--> 222 combined = super().map(func, shortcut=shortcut, args=args, **kwargs)
224 # If the aggregation function didn't drop the original resampling
225 # dimension, then we need to do so before we can rename the proxy
226 # dimension we used.
227 if self._dim in combined.coords:
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:824, in DataArrayGroupByBase.map(self, func, shortcut, args, **kwargs)
822 grouped = self._iter_grouped_shortcut() if shortcut else self._iter_grouped()
823 applied = (maybe_wrap_array(arr, func(arr, *args, **kwargs)) for arr in grouped)
--> 824 return self._combine(applied, shortcut=shortcut)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:843, in DataArrayGroupByBase._combine(self, applied, shortcut)
841 def _combine(self, applied, shortcut=False):
842 """Recombine the applied objects like the original."""
--> 843 applied_example, applied = peek_at(applied)
844 coord, dim, positions = self._infer_concat_args(applied_example)
845 if shortcut:
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/utils.py:194, in peek_at(iterable)
190 """Returns the first value from iterable, as well as a new iterator with
191 the same content as the original iterable
192 """
193 gen = iter(iterable)
--> 194 peek = next(gen)
195 return peek, itertools.chain([peek], gen)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:823, in <genexpr>(.0)
781 """Apply a function to each array in the group and concatenate them
782 together into a new array.
783
(...)
820 The result of splitting, applying and combining this array.
821 """
822 grouped = self._iter_grouped_shortcut() if shortcut else self._iter_grouped()
--> 823 applied = (maybe_wrap_array(arr, func(arr, *args, **kwargs)) for arr in grouped)
824 return self._combine(applied, shortcut=shortcut)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:903, in DataArrayGroupByBase.reduce.<locals>.reduce_array(ar)
902 def reduce_array(ar):
--> 903 return ar.reduce(func, dim, axis, keep_attrs=keep_attrs, **kwargs)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/variable.py:1835, in Variable.reduce(self, func, dim, axis, keep_attrs, keepdims, **kwargs)
1832 keep_attrs = _get_keep_attrs(default=False)
1833 attrs = self._attrs if keep_attrs else None
-> 1835 return Variable(dims, data, attrs=attrs)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/variable.py:305, in Variable.__init__(self, dims, data, attrs, encoding, fastpath)
285 """
286 Parameters
287 ----------
(...)
302 unrecognized encoding items.
303 """
304 self._data = as_compatible_data(data, fastpath=fastpath)
--> 305 self._dims = self._parse_dimensions(dims)
306 self._attrs = None
307 self._encoding = None
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/variable.py:573, in Variable._parse_dimensions(self, dims)
571 dims = tuple(dims)
572 if len(dims) != self.ndim:
--> 573 raise ValueError(
574 f"dimensions {dims} must have the same length as the "
575 f"number of data dimensions, ndim={self.ndim}"
576 )
577 return dims
ValueError: dimensions ('x', 'y') must have the same length as the number of data dimensions, ndim=3
我知道有一种方法DataArray.quantile，但是除了内置方法之外，有没有办法应用如此复杂的归约操作？
啊！使用了错误的函数：P
答案是使用 DataArrayResample.map，它接受将在每个重采样数组组上调用的任意函数：
In [26]: da.resample(time='YS').map(xr.DataArray.quantile, dim='time', q=[0.05, 0.5, 0.95])
Out[26]:
<xarray.DataArray 'temperature' (x: 5, y: 5, time: 111, quantile: 3)>
array([[[[0.0547231 , 0.52834555, 0.90898246],
[0.03937917, 0.47764301, 0.85783451],
[0.1831444 , 0.73754491, 0.95768132],
...,
[0.15764809, 0.45341975, 0.82642675],
[0.08960922, 0.58044495, 0.92426694],
[0.06406972, 0.39973853, 0.93490158]],
[[0.10249662, 0.58991243, 0.86311784],
[0.07144034, 0.4740735 , 0.76378298],
[0.0710782 , 0.31446059, 0.94564097],
...,
[0.23493515, 0.51843013, 0.9406421 ],
[0.0690139 , 0.64156157, 0.93425132],
[0.1194075 , 0.57666303, 0.95710514]],
[[0.15595942, 0.6209935 , 0.98858661],
[0.0207742 , 0.47777671, 0.89507499],
[0.08594763, 0.34327726, 0.80557662],
...,
...
...,
[0.02261331, 0.48502201, 0.7424801 ],
[0.25958214, 0.48240717, 0.70402465],
[0.04444505, 0.43669245, 0.73650377]],
[[0.11350739, 0.44606372, 0.94441655],
[0.1208225 , 0.40995557, 0.95588474],
[0.17862152, 0.58498257, 0.91792651],
...,
[0.05047789, 0.41899782, 0.78520939],
[0.17188789, 0.52498389, 0.89871331],
[0.12420546, 0.54887246, 0.8607523 ]],
[[0.07811254, 0.40114625, 0.83073881],
[0.05883699, 0.49173156, 0.94915052],
[0.19785963, 0.50731951, 0.94814367],
...,
[0.14560478, 0.42114143, 0.85011062],
[0.19936537, 0.58131208, 0.96502682],
[0.11788787, 0.50436407, 0.90359179]]]])
Coordinates:
* time (time) datetime64[ns] 1990-01-01 1991-01-01 ... 2100-01-01
* x (x) float64 -110.0 -105.0 -100.0 -95.0 -90.0
* y (y) float64 23.0 24.75 26.5 28.25 30.0
* quantile (quantile) float64 0.05 0.5 0.95
假设我有一个索引为 (x, y, time) 的数据集：
import xarray as xr, pandas as pd, numpy as np
x = np.linspace(-110, -90, 5)
y = np.linspace(23, 30, 5)
time = pd.date_range('1990-01-01', '2100-12-01', freq='MS')
da = xr.DataArray(
np.random.random(size=(5, 5, len(time))),
dims=['x', 'y', 'time'],
coords=[x, y, time],
name='temperature',
)
有没有办法执行任意重采样或 groupby 缩减，从而沿新维度（分组维度除外）扩展数组？类似于 np.quantile 或 xhistogram.xarray.histogram?
例如，np.quantile 在结果中创建了一个新维度，这会导致错误：
In [15]: da.resample(time='YS').reduce(np.quantile, q=[0.05, 0.5, 0.95])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [15], in <cell line: 1>()
----> 1 da.resample(time='YS').reduce(np.percentile, q=[0.05, 0.5, 0.95])
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:907, in DataArrayGroupByBase.reduce(self, func, dim, axis, keep_attrs, shortcut, **kwargs)
903 return ar.reduce(func, dim, axis, keep_attrs=keep_attrs, **kwargs)
905 check_reduce_dims(dim, self.dims)
--> 907 return self.map(reduce_array, shortcut=shortcut)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/resample.py:222, in DataArrayResample.map(self, func, shortcut, args, **kwargs)
179 """Apply a function to each array in the group and concatenate them
180 together into a new array.
181
(...)
218 The result of splitting, applying and combining this array.
219 """
220 # TODO: the argument order for Resample doesn't match that for its parent,
221 # GroupBy
--> 222 combined = super().map(func, shortcut=shortcut, args=args, **kwargs)
224 # If the aggregation function didn't drop the original resampling
225 # dimension, then we need to do so before we can rename the proxy
226 # dimension we used.
227 if self._dim in combined.coords:
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:824, in DataArrayGroupByBase.map(self, func, shortcut, args, **kwargs)
822 grouped = self._iter_grouped_shortcut() if shortcut else self._iter_grouped()
823 applied = (maybe_wrap_array(arr, func(arr, *args, **kwargs)) for arr in grouped)
--> 824 return self._combine(applied, shortcut=shortcut)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:843, in DataArrayGroupByBase._combine(self, applied, shortcut)
841 def _combine(self, applied, shortcut=False):
842 """Recombine the applied objects like the original."""
--> 843 applied_example, applied = peek_at(applied)
844 coord, dim, positions = self._infer_concat_args(applied_example)
845 if shortcut:
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/utils.py:194, in peek_at(iterable)
190 """Returns the first value from iterable, as well as a new iterator with
191 the same content as the original iterable
192 """
193 gen = iter(iterable)
--> 194 peek = next(gen)
195 return peek, itertools.chain([peek], gen)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:823, in <genexpr>(.0)
781 """Apply a function to each array in the group and concatenate them
782 together into a new array.
783
(...)
820 The result of splitting, applying and combining this array.
821 """
822 grouped = self._iter_grouped_shortcut() if shortcut else self._iter_grouped()
--> 823 applied = (maybe_wrap_array(arr, func(arr, *args, **kwargs)) for arr in grouped)
824 return self._combine(applied, shortcut=shortcut)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/groupby.py:903, in DataArrayGroupByBase.reduce.<locals>.reduce_array(ar)
902 def reduce_array(ar):
--> 903 return ar.reduce(func, dim, axis, keep_attrs=keep_attrs, **kwargs)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/variable.py:1835, in Variable.reduce(self, func, dim, axis, keep_attrs, keepdims, **kwargs)
1832 keep_attrs = _get_keep_attrs(default=False)
1833 attrs = self._attrs if keep_attrs else None
-> 1835 return Variable(dims, data, attrs=attrs)
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/variable.py:305, in Variable.__init__(self, dims, data, attrs, encoding, fastpath)
285 """
286 Parameters
287 ----------
(...)
302 unrecognized encoding items.
303 """
304 self._data = as_compatible_data(data, fastpath=fastpath)
--> 305 self._dims = self._parse_dimensions(dims)
306 self._attrs = None
307 self._encoding = None
File ~/opt/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/variable.py:573, in Variable._parse_dimensions(self, dims)
571 dims = tuple(dims)
572 if len(dims) != self.ndim:
--> 573 raise ValueError(
574 f"dimensions {dims} must have the same length as the "
575 f"number of data dimensions, ndim={self.ndim}"
576 )
577 return dims
ValueError: dimensions ('x', 'y') must have the same length as the number of data dimensions, ndim=3
我知道有一种方法DataArray.quantile，但是除了内置方法之外，有没有办法应用如此复杂的归约操作？
啊！使用了错误的函数：P
答案是使用 DataArrayResample.map，它接受将在每个重采样数组组上调用的任意函数：
In [26]: da.resample(time='YS').map(xr.DataArray.quantile, dim='time', q=[0.05, 0.5, 0.95])
Out[26]:
<xarray.DataArray 'temperature' (x: 5, y: 5, time: 111, quantile: 3)>
array([[[[0.0547231 , 0.52834555, 0.90898246],
[0.03937917, 0.47764301, 0.85783451],
[0.1831444 , 0.73754491, 0.95768132],
...,
[0.15764809, 0.45341975, 0.82642675],
[0.08960922, 0.58044495, 0.92426694],
[0.06406972, 0.39973853, 0.93490158]],
[[0.10249662, 0.58991243, 0.86311784],
[0.07144034, 0.4740735 , 0.76378298],
[0.0710782 , 0.31446059, 0.94564097],
...,
[0.23493515, 0.51843013, 0.9406421 ],
[0.0690139 , 0.64156157, 0.93425132],
[0.1194075 , 0.57666303, 0.95710514]],
[[0.15595942, 0.6209935 , 0.98858661],
[0.0207742 , 0.47777671, 0.89507499],
[0.08594763, 0.34327726, 0.80557662],
...,
...
...,
[0.02261331, 0.48502201, 0.7424801 ],
[0.25958214, 0.48240717, 0.70402465],
[0.04444505, 0.43669245, 0.73650377]],
[[0.11350739, 0.44606372, 0.94441655],
[0.1208225 , 0.40995557, 0.95588474],
[0.17862152, 0.58498257, 0.91792651],
...,
[0.05047789, 0.41899782, 0.78520939],
[0.17188789, 0.52498389, 0.89871331],
[0.12420546, 0.54887246, 0.8607523 ]],
[[0.07811254, 0.40114625, 0.83073881],
[0.05883699, 0.49173156, 0.94915052],
[0.19785963, 0.50731951, 0.94814367],
...,
[0.14560478, 0.42114143, 0.85011062],
[0.19936537, 0.58131208, 0.96502682],
[0.11788787, 0.50436407, 0.90359179]]]])
Coordinates:
* time (time) datetime64[ns] 1990-01-01 1991-01-01 ... 2100-01-01
* x (x) float64 -110.0 -105.0 -100.0 -95.0 -90.0
* y (y) float64 23.0 24.75 26.5 28.25 30.0
* quantile (quantile) float64 0.05 0.5 0.95