4

I have multiple data with the same coords and similar values for them. I.e., there is no single unique ID but rather a combined ID of (index, split). Ideally, I would just want to append all the datasets one after another, but I haven't found the right way to do so.

import xarray as xr
from xarray import Dataset
import numpy as np

datasets = []
for split in range(3):
    dim2_len = 4
    dim1_len = 3
    data_len = 5
    d = Dataset({'data1': (['index', 'dim1'], np.random.rand(data_len, dim1_len)),
                 'data2': (['index', 'dim2'], np.random.rand(data_len, dim2_len)),
                 'data3': (['index', 'dim2'], np.random.rand(data_len, dim2_len))},
                coords={'index': range(data_len),
                        'dim2': range(dim2_len),
                        'dim1': range(dim1_len),
                        'split': split})
    datasets.append(d)

xr.auto_combine(datasets)
# ValueError: too many different dimensions to concatenate: {'dim1', 'dim2', 'index'}

The current xr.concat / xr.auto_combine function does not allow you to concatenate datasets along multiple dimensions (https://github.com/pydata/xarray/blob/4b8339b53f1b9dcd79f2a9060933713328a13b90/xarray/core/combine.py#L358). I guess, I could manually take all the data and coords and throw them into a new dataset but it seems like there should be a better way in the library.

Is there a way to concatenate datasets along multiple dimensions?

mschrimpf
  • 539
  • 2
  • 8
  • 19
  • Can you provide some code explaining what you've tried and a summary of how your datasets are structured? This question is not complete in its current form. – jhamman Nov 29 '17 at 16:58
  • 1
    I just answered a similar question: https://stackoverflow.com/questions/47545138/python-xarray-open-mfdataset-concat-along-two-dimensions, can you try breaking your `concat` operations into pieces, one dimension at a time? – jhamman Nov 29 '17 at 23:55

0 Answers0