-
-
Notifications
You must be signed in to change notification settings - Fork 18.7k
BUG: to_numeric fails to convert a Pyarrow Decimal series containing NA values #61659
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -919,3 +919,14 @@ def test_coerce_pyarrow_backend(): | |||||
result = to_numeric(ser, errors="coerce", dtype_backend="pyarrow") | ||||||
expected = Series([1, 2, None], dtype=ArrowDtype(pa.int64())) | ||||||
tm.assert_series_equal(result, expected) | ||||||
|
||||||
|
||||||
def test_to_numeric_arrow_decimal_with_na(): | ||||||
# GH 61641 | ||||||
pa = pytest.importorskip("pyarrow") | ||||||
decimal_type = ArrowDtype(pa.decimal128(3, scale=2)) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe we could add the dtype as parameterization and also add any other EA dtypes that are expected to no-op? so we could call the test something like.. test_to_numeric_EA_is_numeric? |
||||||
series = Series([1, None], dtype=decimal_type) | ||||||
result = to_numeric(series, errors="coerce") | ||||||
|
||||||
expected = Series([1.00, pd.NA], dtype=decimal_type) | ||||||
tm.assert_series_equal(result, expected) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IIUC we want a no-op here? so does
Suggested change
work? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you try to specify
if not is_numeric_dtype
here, and let the logic below try to convert this case anyways? This short circuit would skip some of the options set into_numeric
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also as @mroeschke suggested in #61641 (comment) documentation update would also be welcome