.sel does not keep selected coordinate value in case with MultiIndex

[fujiisoup]

.sel method usually keeps selected coordinate value as a scalar coordinate

In[4] ds1 = xr.Dataset({'foo': (('x',), [1, 2, 3])}, {'x': [1, 2, 3], 'y': 'a'})

Out[4]: 
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    y        <U1 'a'
    x        int64 1
Data variables:
    foo      int64 1

But in MultiIndex case, does not.

In[5] ds2 = xr.Dataset({'foo': (('x',), [4, 5, 6])}, {'x': [1, 2, 3], 'y': 'b'})
        ds = xr.concat([ds1, ds2], dim='y').stack(yx=['y', 'x'])
        ds.sel(x=2)
Out[5] 
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
Data variables:
    foo      (y) int64 2 5

x is gone.
Is it a desired behavior?

[shoyer]

No, I don't think this was intentional/desired, though it might be slightly tricky to fix this with the current code structure.

@benbovy any thoughts?

For reference:

In [4]: ds
Out[4]:
<xarray.Dataset>
Dimensions:  (yx: 6)
Coordinates:
  * yx       (yx) MultiIndex
  - y        (yx) object 'a' 'a' 'a' 'b' 'b' 'b'
  - x        (yx) int64 1 2 3 1 2 3
Data variables:
    foo      (yx) int64 1 2 3 4 5 6

In [5]: ds.sel(x=2)
Out[5]:
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
Data variables:
    foo      (y) int64 2 5

[fujiisoup]

@shoyer Thanks for the information.
(Sorry for my inaccurate ipython output)

Another related issue is

In[3]: ds.isel(yx=1)
Out[3]: 
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    yx       object ('a', 1)
Data variables:
    foo      int64 1

where MultiIndex structure is totally gone.
(ds.isel(yx=[0, 1]) works perfectly.)

I am now looking into the code. Do you guys have any idea to fix them?
I am thinking to modify Index.remap_label_indexers
so that it also returns selected_dims containing names of dims that should be converted to scalar coordinates.

[benbovy]

The current behavior is actually dictated by pandas and I think it was initially intentional (before we could access MultiIndex levels as virtual coordinates).

In the first example MultiIndex.get_loc_level() is called, which doesn't return a MutliIndex:

In [11]: mdix = ds.yx.to_index()

In [12]: midx.get_loc_level(2, level='x')
Out[12]: 
(array([False,  True, False, False,  True, False], dtype=bool),
 Index(['a', 'b'], dtype='object', name='y'))

If we want to keep the MultiIndex in this case, then I think we can just ignore new_indexes returned by indexing.remap_label_indexers and remove Dataset._replace_indexes.

But we still have to choose whether automatically renaming the dimension makes sense or not, i.e., choose between

In [5]: ds.sel(x=2)
Out[5]:
<xarray.Dataset>
Dimensions:  (yx: 2)
Coordinates:
  * yx       MultiIndex 
  - y        (yx) object 'a' 'b'
  - x        (yx) int64 2 2
Data variables:
    foo      (yx) int64 2 5

and

In [5]: ds.sel(x=2)
Out[5]:
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
    x        (y) int64 2 2
Data variables:
    foo      (y) int64 2 5

---

The second issue is also due to how MultiIndex works in pandas:

In [10]: midx[0]
Out[10]: ('a', 1)

In [11]: midx[[0, 1]]
Out[11]: 
MultiIndex(levels=[['a', 'b'], [1, 2, 3]],
           labels=[[0, 0], [0, 1]],
           names=['y', 'x'])

I'm just wondering if it really makes sense to keep a MultiIndex for a coordinate that is not a dimension anymore like in this case.

[fujiisoup]

@benbovy Thanks for the comment.

Actually, I expected the following behavior

In [1]: ds.sel(x=2)
Out[1]: 
<xarray.Dataset>
Dimensions:  (y: 2)
Coordinates:
  * y        (y) object 'a' 'b'
    x        int64 2
Data variables:
    foo      (y) int64 2 5

where the selected x is automatically converted to a scalar coordinate.

And also for the second issue, I expected something like

In [2]: ds.isel(yx=0)
Out[2]: 
<xarray.Dataset>
Dimensions:  ()
Coordinates:
    y        <U1 'a'
    x        int64 1
Data variables:
    foo      int64 1

both x and y become scalar coordinates.

Back to the first issue, I personally prefer to the automatic renaming
because it enables the following chain methods,

ds.sel(x=2).isel(y=1)

that might be intuitive especially for the hierarchic dimension structure.

[benbovy]

(mmh apparently I forgot to post a 2nd comment I wrote)

I'm just wondering if it really makes sense to keep a MultiIndex for a coordinate that is not a dimension anymore like in this case.

And this is just a nonsense :) Sorry it is late!

Yep I agree it would be nice to have scalar coordinates in both cases.

[stale]

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

[gabriel-abrahao]

As a related issue, we could use an option for .sel to keep the selected variable as a singleton coordinate (an array of shape (1,)) instead of making it a scalar.