ExtensionArray meta-issue

[TomAugspurger]

Just so things don't get lost

• Interface, ExtensionBlock, and Internals (Array Interface and Categorical internals Refactor #19268)
• Series and DataFrame can hold extension arrays (ENH: Allow storing ExtensionArrays in containers #19520)
• overridable assert_*_equal (REF: Base class for all extension tests #19863)
• unique (ENH: ExtensionArray.unique #19869)
• fillna (ENH: ExtensionArray.fillna #19909)
• __setitem__ works properly (ENH: ExtensionArray.setitem #19907)
• Document everything: (DOC: add doc on ExtensionArray and extending pandas #19936)
• argsort, sort_values (ENH: Sorting of ExtensionArrays #19957)
• factorize (ENH/API: ExtensionArray.factorize #20361)
• groupby works properly (blocked by factorize)
• IntervalArray: (https://github.com/TomAugspurger/pandas/tree/pandas-array-upstream%2Bfu1%2Binterval)
• PeriodArray: index backed by PerioidArray, store in Series / Frame
• Refactor arithmetic methods from DatetimeIndexOpsMixin/DatetimeIndex/TimedeltaIndex/PeriodIndex into array classes (ENH: implement DatetimeLikeArray #19902)
• Refactor comparison methods from DTI/TDI/PI into array classes
• select_dtypes

[jbrockmendel]

Should these have a _typ attribute? If so, should SparseArray._typ be changed from "array" to something more specific?

[TomAugspurger]

We're adding an ABCExtensionArray and `EA._type` with the value 'extension' in #19520. SparseArray being _array isn't the perfect name, but I don't think that matters, do you? The more important name is the ABC.

…

On Wed, Feb 21, 2018 at 11:37 PM, jbrockmendel ***@***.***> wrote: Should these have a _typ attribute? If so, should SparseArray._typ be changed from "array" to something more specific? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19696 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIkHC59w-YBE7ATTy1l6gtL-GgsSgks5tXP0MgaJpZM4SFUFN> .

[TomAugspurger]

Quick status update. The two big ones are in, and now work can mostly proceed in parallel (aside from #19863 which a couple of my followups depend on).

I'm currently rebasing all my branches on master and will make PRs today.

I'll also go through all the TODOs we've added an make issues / checkboxes for those.

If anyone is interested in working on these, PeriodArray will likely be the most sizable PR that I don't really have a start on, aside from a now out of date branch.

[jbrockmendel]

re PeriodArray, were in spitting distance of being ready to refactor arith/cmp methods out from DatetimeIndexOpsMixin/DatetimeIndexTimedeltaIndex/PeriodIndex. If you can wait for #19847, #19835, and #19800, I can get the ball rolling and save you some work.

[jbrockmendel]

• Refactor arithmetic methods from DatetimeIndexOpsMixin/DatetimeIndex/TimedeltaIndex/PeriodIndex into array classes (ENH: implement DatetimeLikeArray #19902)
• Refactor comparison methods from DTI/TDI/PI into array classes

[TomAugspurger]

Porting from #19902 (comment)

When do we want to call it and release 0.23? I can get __setitem__ finished up tonight or tomorrow. Groupby is I think close to being done (we have the API issue for argsort and a similar one for factorize. Actually supporting groupby doesn't seem difficult). Once those are done, I would call the interface sufficiently complete (aside perhaps from support for arithmetic ops, which I haven't really looked into. I'm not sure how difficult that will be, but I imagine it'll be some effort).

The big remaining question then is moving other array types to EAs

• Period
• Interval
• DatetimeTZ
• Sparse

Do we want to block 0.23 for those? Do we have an estimate for how long that would take?

cc @shoyer @jreback @jorisvandenbossche @chris-b1.

[jbrockmendel]

Do we have an estimate for how long that would take?

For the datetime/timedelta/period arrays (or the arithmetic/comparison bits of them), if/when #19902 goes through, I can get the next two PRs in that sequence out pretty quickly. But after those are in there will be some non-trivial design decisions to be made. The end of March would be an optimistic goal.