-
-
Notifications
You must be signed in to change notification settings - Fork 18.7k
Closed
Labels
PerformanceMemory or execution speed performanceMemory or execution speed performanceSparseSparse Data TypeSparse Data Type
Milestone
Description
This currently densifies:
pandas/pandas/core/sparse/array.py
Lines 651 to 659 in 56d8e78
# TODO: I think we can avoid densifying when masking a | |
# boolean SparseArray with another. Need to look at the | |
# key's fill_value for True / False, and then do an intersection | |
# on the indicies of the sp_values. | |
if isinstance(key, SparseArray): | |
if is_bool_dtype(key): | |
key = key.to_dense() | |
else: | |
key = np.asarray(key) |
I haven't investigated it, but we should be able to do a boolean mask as an
intersection sp_values on self
and key
. If key
is SparseDtype[bool, False]
(i.e. False is the fill_value) this should be a lot faster.
Metadata
Metadata
Assignees
Labels
PerformanceMemory or execution speed performanceMemory or execution speed performanceSparseSparse Data TypeSparse Data Type