Skip to content

fix pkg_resources DeprecationWarning #40548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from

Conversation

cxzhong
Copy link
Contributor

@cxzhong cxzhong commented Aug 6, 2025

Fixes #38389. I think it will fix the pkg_resources Deprecation Warning when upgrading setuptools. Because from setuptools 80.9.0, it does not reply on the pkg_resources

/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/sage-venv-config:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html

  __import__('pkg_resources').require('sagemath-standard==10.7rc0')

/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/sage-ipython:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html

  __import__('pkg_resources').require('sagemath-standard==10.7rc0')

📝 Checklist

  • The title is concise and informative.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation and checked the documentation preview.

⌛ Dependencies

@cxzhong cxzhong marked this pull request as ready for review August 6, 2025 15:08
Copy link

github-actions bot commented Aug 6, 2025

Documentation preview for this PR (built with commit 3e88fc6; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

@cxzhong cxzhong marked this pull request as draft August 7, 2025 06:02
@cxzhong cxzhong marked this pull request as ready for review August 7, 2025 10:55
@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 7, 2025

Since, after that I found the doctest crashed when gap/elements.pyx. I just changed some context of doctests. when runs

libgap.Sum(*rnd).

The python crashed

GNU gdb (Ubuntu 16.2-8ubuntu1) 16.2
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
(No debugging symbols found in /usr/bin/python3.13)
(No debugging symbols found in /lib/x86_64-linux-gnu/libz.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libexpat.so.1)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_bz2.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libbz2.so.1.0)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_lzma.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/liblzma.so.5)
(No debugging symbols found in /home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/signals.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgcc_s.so.1)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_hashlib.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libcrypto.so.3)
(No debugging symbols found in /lib/x86_64-linux-gnu/libzstd.so.1)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_ssl.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libssl.so.3)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_ctypes.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libffi.so.8)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/mmap.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_multiprocessing.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/readline.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libreadline.so.8)
(No debugging symbols found in /lib/x86_64-linux-gnu/libtinfo.so.6)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_queue.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_decimal.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/termios.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/resource.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_sqlite3.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libsqlite3.so.0)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_contextvars.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_asyncio.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_uuid.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libuuid.so.1)
(No debugging symbols found in /usr/lib/python3.13/lib-dynload/_lsprof.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgmp.so.10)
(No debugging symbols found in /lib/x86_64-linux-gnu/libmpfr.so.6)
(No debugging symbols found in /lib/x86_64-linux-gnu/libmpfi.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libmpc.so.3)
(No debugging symbols found in /lib/x86_64-linux-gnu/libpari-gmp-tls.so.9)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgsl.so.28)
(No debugging symbols found in /lib/x86_64-linux-gnu/libopenblas.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgslcblas.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgfortran.so.5)
(No debugging symbols found in /lib/x86_64-linux-gnu/libflint.so.19)
(No debugging symbols found in /lib/x86_64-linux-gnu/libstdc++.so.6)
Function(s) ^std::(move|forward|as_const|(__)?addressof) will be skipped when stepping.
Function(s) ^std::(shared|unique)_ptr<.*>::(get|operator) will be skipped when stepping.
Function(s) ^std::(basic_string|vector|array|deque|(forward_)?list|(unordered_|flat_)?(multi)?(map|set)|span)<.*>::(c?r?(begin|end)|front|back|data|size|empty) will be skipped when stepping.
Function(s) ^std::(basic_string|vector|array|deque|span)<.*>::operator.] will be skipped when stepping.
(No debugging symbols found in /lib/x86_64-linux-gnu/libtiff.so.6)
(No debugging symbols found in /lib/x86_64-linux-gnu/libjpeg.so.8)
(No debugging symbols found in /lib/x86_64-linux-gnu/libimagequant.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libxcb.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libwebp.so.7)
(No debugging symbols found in /lib/x86_64-linux-gnu/libLerc.so.4)
(No debugging symbols found in /lib/x86_64-linux-gnu/libjbig.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libdeflate.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgomp.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libXau.so.6)
(No debugging symbols found in /lib/x86_64-linux-gnu/libXdmcp.so.6)
(No debugging symbols found in /lib/x86_64-linux-gnu/libsharpyuv.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libfreetype.so.6)
(No debugging symbols found in /lib/x86_64-linux-gnu/libpng16.so.16)
(No debugging symbols found in /lib/x86_64-linux-gnu/libbrotlidec.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libbrotlicommon.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libntl.so.44)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgf2x.so.3)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgmpxx.so.4)
(No debugging symbols found in /home/zhongcx/sage/local/lib/libSingular-4.4.1.so)
(No debugging symbols found in /home/zhongcx/sage/local/lib/libpolys-4.4.1.so)
(No debugging symbols found in /home/zhongcx/sage/local/lib/libfactory-4.4.1.so)
(No debugging symbols found in /home/zhongcx/sage/local/lib/libsingular_resources-4.4.1.so)
(No debugging symbols found in /home/zhongcx/sage/local/bin/../lib/singular/MOD/singmathic.so)
(No debugging symbols found in /home/zhongcx/sage/local/bin/../lib/singular/MOD/p_Procs_FieldIndep.so)
(No debugging symbols found in /lib/x86_64-linux-gnu/libiml.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libblas.so.3)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgd.so.3)
(No debugging symbols found in /lib/x86_64-linux-gnu/libraqm.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libfontconfig.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libXpm.so.4)
(No debugging symbols found in /lib/x86_64-linux-gnu/libheif.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libharfbuzz.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libfribidi.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libX11.so.6)
(No debugging symbols found in /lib/x86_64-linux-gnu/libglib-2.0.so.0)
(No debugging symbols found in /lib/x86_64-linux-gnu/libgraphite2.so.3)
(No debugging symbols found in /lib/x86_64-linux-gnu/libatomic.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/libpcre2-8.so.0)
(No debugging symbols found in /lib/libcliquer.so.1)
(No debugging symbols found in /lib/x86_64-linux-gnu/librw.so.0)
(No debugging symbols found in /home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/alarm.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/pselect.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/pysignals.cpython-313-x86_64-linux-gnu.so)
(No debugging symbols found in /home/zhongcx/sage/local/bin/../lib/singular/MOD/p_Procs_FieldGeneral.so)
(No debugging symbols found in /home/zhongcx/sage/local/bin/../lib/singular/MOD/p_Procs_FieldQ.so)
(No debugging symbols found in /home/zhongcx/sage/local/bin/../lib/singular/MOD/p_Procs_FieldZp.so)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
__syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S:56

Stack backtrace
---------------
No locals.
#1  0x00007f89c1165b63 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, 
    a4=<optimized out>, a5=0, a6=0, nr=61) at ./nptl/cancellation.c:49
        result = <optimized out>
        pd = <optimized out>
        ch = <optimized out>
        result = <optimized out>
        pd = <optimized out>
        ch = <optimized out>
        resultvar = <optimized out>
        __arg6 = <optimized out>
        __arg5 = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a6 = <optimized out>
        _a5 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#2  __syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, a4=<optimized out>, a5=a5@entry=0, 
    a6=a6@entry=0, nr=61) at ./nptl/cancellation.c:75
        r = <optimized out>
#3  0x00007f89c11e1fdf in __GI___wait4 (pid=<optimized out>, stat_loc=<optimized out>, options=<optimized out>, 
    usage=<optimized out>) at ../sysdeps/unix/sysv/linux/wait4.c:30
No locals.
#4  0x00007f89c0b4afab in sigdie ()
   from /home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/signals.cpython-313-x86_64-linux-gnu.so
No symbol table info available.
#5  0x00007f89c0b4cf81 in cysigs_signal_handler ()
   from /home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/signals.cpython-313-x86_64-linux-gnu.so
No symbol table info available.
#6  <signal handler called>
No locals.
#7  __syscall_cancel_arch () at ../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S:56
No locals.
#8  0x00007f89c1165ae3 in __internal_syscall_cancel (a1=<optimized out>, a2=<optimized out>, a3=<optimized out>, 
    a4=<optimized out>, a5=a5@entry=0, a6=a6@entry=0, nr=230) at ./nptl/cancellation.c:49
        result = <optimized out>
        pd = <optimized out>
        ch = <optimized out>
#9  0x00007f89c11bb932 in __GI___clock_nanosleep (clock_id=<optimized out>, flags=<optimized out>, 
    req=<optimized out>, rem=<optimized out>) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:48
        r = <optimized out>
#10 0x00000000006b463e in ?? ()
No symbol table info available.
#11 0x000000000058b6cc in ?? ()
No symbol table info available.
#12 0x00000000005505e3 in PyObject_Vectorcall ()
No symbol table info available.
#13 0x0000000000566c35 in _PyEval_EvalFrameDefault ()
No symbol table info available.
#14 0x000000000065c91d in PyEval_EvalCode ()
No symbol table info available.
#15 0x0000000000677845 in ?? ()
No symbol table info available.
#16 0x0000000000566251 in _PyEval_EvalFrameDefault ()
No symbol table info available.
#17 0x000000000065a784 in ?? ()
No symbol table info available.
#18 0x00000000006b0405 in ?? ()
No symbol table info available.
#19 0x000000000054aab3 in _PyObject_MakeTpCall ()
No symbol table info available.
#20 0x000000000056e85f in _PyEval_EvalFrameDefault ()
No symbol table info available.
#21 0x000000000065c91d in PyEval_EvalCode ()
No symbol table info available.
#22 0x0000000000677845 in ?? ()
No symbol table info available.
#23 0x0000000000566251 in _PyEval_EvalFrameDefault ()
No symbol table info available.
#24 0x000000000065a784 in ?? ()
No symbol table info available.
#25 0x00000000006b0405 in ?? ()
No symbol table info available.
#26 0x000000000054aab3 in _PyObject_MakeTpCall ()
No symbol table info available.
#27 0x000000000056e85f in _PyEval_EvalFrameDefault ()
No symbol table info available.
#28 0x00000000005a3bf5 in ?? ()
No symbol table info available.
#29 0x000000000054aa0f in _PyObject_MakeTpCall ()
No symbol table info available.
#30 0x0000000000566c35 in _PyEval_EvalFrameDefault ()
No symbol table info available.
#31 0x000000000065c91d in PyEval_EvalCode ()
No symbol table info available.
#32 0x000000000067d6e2 in ?? ()
No symbol table info available.
#33 0x000000000067961e in ?? ()
No symbol table info available.
#34 0x0000000000692d94 in ?? ()
No symbol table info available.
#35 0x0000000000692675 in ?? ()
No symbol table info available.
#36 0x00000000006924b5 in ?? ()
No symbol table info available.
#37 0x000000000069126a in Py_RunMain ()
No symbol table info available.
#38 0x000000000064bc8d in Py_BytesMain ()
No symbol table info available.
#39 0x00007f89c10f1578 in __libc_start_call_main (main=main@entry=0x64bbd0, argc=argc@entry=4, 
    argv=argv@entry=0x7fffdbf9d1f8) at ../sysdeps/nptl/libc_start_call_main.h:58
        self = <optimized out>
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736883970552, -2937273851686401557, 4, 0, 10423760, 
                140229629931520, -2937273851705275925, -2895672641550661141}, mask_was_saved = 0}}, priv = {pad = {
              0x0, 0x0, 0x64b1db, 0x9f0dc8}, data = {prev = 0x0, cleanup = 0x0, canceltype = 6599131}}}
        not_first_call = <optimized out>
#40 0x00007f89c10f163b in __libc_start_main_impl (main=0x64bbd0, argc=4, argv=0x7fffdbf9d1f8, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffdbf9d1e8) at ../csu/libc-start.c:360
No locals.
#41 0x000000000064b035 in _start ()
No symbol table info available.
[Inferior 1 (process 632) detached]
warning: 56	../sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S: No such file or directory
Python Exception <class 'ModuleNotFoundError'>: No module named 'Cython'
Error occurred in Python: No module named 'Cython'

@cxzhong cxzhong changed the title fixes pkg_resources DeprecationWarning fix pkg_resources DeprecationWarning Aug 10, 2025
@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 11, 2025

What was accomplished:
Fixed the root cause: Enhanced exception handling in GapElement_Function.__call__ method in element.pyx

Key improvements:

Added sig_on_called flag to track signal handling state
Proper cleanup of signals when exceptions occur
Ensured GAPError exceptions are properly raised instead of segfaults
Updated doctest to import GAPError correctly
Thorough testing:

All 523 tests in element.pyx pass
Original segfault case now properly raises GAPError
Normal GAP function usage continues to work correctly
No regressions in GAP library functionality
Successfully committed and pushed:

Commit hash: de8b7ad
Branch: update-setuptools
Pushed to remote repository
The fix resolves:
Segmentation faults when calling libgap.Sum(*[1,2,3]) and similar incorrect function calls
Issues caused by upgraded setuptools, beniget, and gast packages
Broken error handling in GAP function interface

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 11, 2025

@orlitzky I have completed this PR. Can you help me review it? Thank you very much.

@orlitzky
Copy link
Contributor

I think your GAP issue is the same as #37026. It does not happen consistently, so if you change the test to expect a GAPError instead of a ValueError, then the test will fail on machines where there is no crash (because a ValueError will still be raised). In any case, the test should not pass if GAP crashes, because GAP is not supposed to crash here :)

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 11, 2025

I think your GAP issue is the same as #37026. It does not happen consistently, so if you change the test to expect a GAPError instead of a ValueError, then the test will fail on machines where there is no crash (because a ValueError will still be raised). In any case, the test should not pass if GAP crashes, because GAP is not supposed to crash here :)

So can I catch GAPError and ValueError both?
I just do the sig_on() and sig_off() in the exception handle. And I think it will not lead to segfault now.

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 11, 2025

I think your GAP issue is the same as #37026. It does not happen consistently, so if you change the test to expect a GAPError instead of a ValueError, then the test will fail on machines where there is no crash (because a ValueError will still be raised). In any case, the test should not pass if GAP crashes, because GAP is not supposed to crash here :)

@orlitzky My gap is 4.14. Still happens that in sagemath 10.7 But before updating 3 packages, I will get GAPError. After that I will get segfault. And after the commit, the result return to GAPError.

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 11, 2025

I think your GAP issue is the same as #37026. It does not happen consistently, so if you change the test to expect a GAPError instead of a ValueError, then the test will fail on machines where there is no crash (because a ValueError will still be raised). In any case, the test should not pass if GAP crashes, because GAP is not supposed to crash here :)

@orlitzky You can find that GAPError does not means GAP has crashed.

    If Gap finds an error while evaluating, a :exc:`GAPError`
    exception is raised::

        sage: libgap.eval('1/0')
        Traceback (most recent call last):
        ...
        GAPError: Error, Rational operations: <divisor> must not be zero

    Also, a :exc:`GAPError` is raised if the input is not a simple expression::

        sage: libgap.eval('1; 2; 3')
        Traceback (most recent call last):
        ...
        GAPError: can only evaluate a single statement

And I think it is the error message gap returned

@orlitzky
Copy link
Contributor

So can I catch GAPError and ValueError both? I just do the sig_on() and sig_off() in the exception handle. And I think it will not lead to segfault now.

We want it to lead to a segfault though. There is an unfixed bug in Sage. You can catch the segfault and turn it into a GAPError, and then update the test to catch the GAPError, but all you've done is make it so that the test suite no longer detects the problem.

It is going to be hard to solve correctly because it involves both the C code generated by cython and cysignals. There is a great explanation here: https://trofi.github.io/posts/312-the-sagemath-saga.html

Basically, we need to add volatile to a lot of variables when we are using sig_on() and sig_off(), but those variables are being automatically generated by Cython so it is not clear how to proceed. There is some further discussion at sagemath/cysignals#194

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 12, 2025

So can I catch GAPError and ValueError both? I just do the sig_on() and sig_off() in the exception handle. And I think it will not lead to segfault now.

We want it to lead to a segfault though. There is an unfixed bug in Sage. You can catch the segfault and turn it into a GAPError, and then update the test to catch the GAPError, but all you've done is make it so that the test suite no longer detects the problem.

It is going to be hard to solve correctly because it involves both the C code generated by cython and cysignals. There is a great explanation here: https://trofi.github.io/posts/312-the-sagemath-saga.html

Basically, we need to add volatile to a lot of variables when we are using sig_on() and sig_off(), but those variables are being automatically generated by Cython so it is not clear how to proceed. There is some further discussion at sagemath/cysignals#194

@orlitzky Thank you for your reply. I just change GAPError to ValueError, and add some volatile. Can you check it again?

@cxzhong cxzhong changed the title fix pkg_resources DeprecationWarning fix pkg_resources DeprecationWarning and the Segmentation fault of element.pyx Aug 12, 2025
@cxzhong cxzhong changed the title fix pkg_resources DeprecationWarning and the Segmentation fault of element.pyx fix pkg_resources DeprecationWarning and the Segmentation fault on element.pyx Aug 12, 2025
@cxzhong cxzhong changed the title fix pkg_resources DeprecationWarning and the Segmentation fault on element.pyx fix pkg_resources DeprecationWarning Aug 13, 2025
@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 13, 2025

@orlitzky I have rolled back the element.pyx. And I just update setuptools, setuptools_scm, gast, beniget, pythran to make them compatible. Now, there are no pkg_resources warnings when start sage as setup 20 in Readme.md Maybe I will split a new PR to see element.pyx. Thank you very much for helping me find the problems.

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 13, 2025

So can I catch GAPError and ValueError both? I just do the sig_on() and sig_off() in the exception handle. And I think it will not lead to segfault now.

We want it to lead to a segfault though. There is an unfixed bug in Sage. You can catch the segfault and turn it into a GAPError, and then update the test to catch the GAPError, but all you've done is make it so that the test suite no longer detects the problem.

It is going to be hard to solve correctly because it involves both the C code generated by cython and cysignals. There is a great explanation here: https://trofi.github.io/posts/312-the-sagemath-saga.html

Basically, we need to add volatile to a lot of variables when we are using sig_on() and sig_off(), but those variables are being automatically generated by Cython so it is not clear how to proceed. There is some further discussion at sagemath/cysignals#194

Finally, I found the problem. It might be a bug in Cython. I upgrade Cython to 3.1.3. Then The problem is solved.

Doctesting 1 file.
sage -t --warn-long 5.0 --random-seed=90585052024886705204440182278073812396 src/sage/libs/gap/element.pyx
    [520 tests, 2.49s wall]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 7.5 seconds
    cpu time: 2.5 seconds
    cumulative wall time: 2.5 seconds

@orlitzky Thank you very much for helping. Now, I think all problems have been solved

@orlitzky
Copy link
Contributor

@orlitzky I have rolled back the element.pyx. And I just update setuptools, setuptools_scm, gast, beniget, pythran to make them compatible. Now, there are no pkg_resources warnings when start sage as setup 20 in Readme.md Maybe I will split a new PR to see element.pyx. Thank you very much for helping me find the problems.

I'm glad, I was going to suggest this yesterday but got busy and ran out of time.

Finally, I found the problem. It might be a bug in Cython. I upgrade Cython to 3.1.3. Then The problem is solved.

I think it is just hidden for now. The segfault happens if some important memory is overwritten after longjmp(), but the location of that memory depends on internal details of cython and the C compiler. Upgrading cython most likely rearranged your local variables so that something less important is overwritten. This will be a much harder problem to pinpoint and solve, though, so it's a good idea to do the easy part (pkg_resources warnings) separately.

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 13, 2025

@orlitzky I have rolled back the element.pyx. And I just update setuptools, setuptools_scm, gast, beniget, pythran to make them compatible. Now, there are no pkg_resources warnings when start sage as setup 20 in Readme.md Maybe I will split a new PR to see element.pyx. Thank you very much for helping me find the problems.

I'm glad, I was going to suggest this yesterday but got busy and ran out of time.

Finally, I found the problem. It might be a bug in Cython. I upgrade Cython to 3.1.3. Then The problem is solved.

I think it is just hidden for now. The segfault happens if some important memory is overwritten after longjmp(), but the location of that memory depends on internal details of cython and the C compiler. Upgrading cython most likely rearranged your local variables so that something less important is overwritten. This will be a much harder problem to pinpoint and solve, though, so it's a good idea to do the easy part (pkg_resources warnings) separately.

It seems that I test successfully on my computer. But the workflow has failed

@orlitzky
Copy link
Contributor

Only one of the failures so far looks related to your changes, and I don't know if it's something we have to worry about. It's failing to download the setuptools tarball because the SSL certificate is not recognized as valid.

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 13, 2025

Only one of the failures so far looks related to your changes, and I don't know if it's something we have to worry about. It's failing to download the setuptools tarball because the SSL certificate is not recognized as valid.

yes. The element.pyx continuously reports segfault. Maybe the problem is in the complier. And I think maybe we can add an exception handle to catch it

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 13, 2025

@orlitzky I have rolled back the element.pyx. And I just update setuptools, setuptools_scm, gast, beniget, pythran to make them compatible. Now, there are no pkg_resources warnings when start sage as setup 20 in Readme.md Maybe I will split a new PR to see element.pyx. Thank you very much for helping me find the problems.

I'm glad, I was going to suggest this yesterday but got busy and ran out of time.

Finally, I found the problem. It might be a bug in Cython. I upgrade Cython to 3.1.3. Then The problem is solved.

I think it is just hidden for now. The segfault happens if some important memory is overwritten after longjmp(), but the location of that memory depends on internal details of cython and the C compiler. Upgrading cython most likely rearranged your local variables so that something less important is overwritten. This will be a much harder problem to pinpoint and solve, though, so it's a good idea to do the easy part (pkg_resources warnings) separately.

Yes. it seems you are right. Upgrading Cython just hides this problem. Maybe I can try to think about it. And Maybe I can put a new PR about element.pyx. In my enviorment it pass all doctests except lazy import of fricas. I think it can be merged. Thank you very much.

@orlitzky
Copy link
Contributor

The rest looks OK to me, but please do a git rebase -i develop to prune all of the intermediate commits. Otherwise the files (like element.pyx) that were modified and un-modified need to be recompiled for no reason.

- Update setuptools to latest version
- Update setuptools_scm to latest version
- Update Cython to version 3.1.3
- Update Pythran to version 0.18.0
- Update beniget and gast package versions
- Fix various build and test issues
@cxzhong cxzhong force-pushed the update-setuptools branch from e06cb4f to 4811722 Compare August 14, 2025 02:41
@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 14, 2025

The rest looks OK to me, but please do a git rebase -i develop to prune all of the intermediate commits. Otherwise the files (like element.pyx) that were modified and un-modified need to be recompiled for no reason.

Yes, everything is OK now. @orlitzky I just try a Test PR to watch the workflow, but it still fails. Maybe it is much harder than I think

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 14, 2025

The rest looks OK to me, but please do a git rebase -i develop to prune all of the intermediate commits. Otherwise the files (like element.pyx) that were modified and un-modified need to be recompiled for no reason.

The problem is still. but not consistent.

sage: libgap.Sum(*[1,2,3])
---------------------------------------------------------------------------
GAPError                                  Traceback (most recent call last)
Cell In[1], line 1
----> 1 libgap.Sum(*[Integer(1),Integer(2),Integer(3)])

File sage/libs/gap/element.pyx:2509, in sage.libs.gap.element.GapElement_Function.__call__()

GAPError: Error, no method found! Error, no 1st choice method found for `SumOp' on 3 arguments
sage: libgap.Sum(*[1,2,3])
---------------------------------------------------------------------------
GAPError                                  Traceback (most recent call last)
Cell In[2], line 1
----> 1 libgap.Sum(*[Integer(1),Integer(2),Integer(3)])

File sage/libs/gap/element.pyx:2509, in sage.libs.gap.element.GapElement_Function.__call__()

GAPError: Error, no method found! Error, no 1st choice method found for `SumOp' on 3 arguments
sage: libgap.Sum(*[1,2,3])
---------------------------------------------------------------------------
GAPError                                  Traceback (most recent call last)
Cell In[3], line 1
----> 1 libgap.Sum(*[Integer(1),Integer(2),Integer(3)])

File sage/libs/gap/element.pyx:2509, in sage.libs.gap.element.GapElement_Function.__call__()

GAPError: Error, no method found! Error, no 1st choice method found for `SumOp' on 3 arguments
sage: libgap.Sum(*[1,2,3])
------------------------------------------------------------------------
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/signals.cpython-313-x86_64-linux-gnu.so(+0x97bf) [0x7b38b49f17bf]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/signals.cpython-313-x86_64-linux-gnu.so(+0x9885) [0x7b38b49f1885]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages/cysignals/signals.cpython-313-x86_64-linux-gnu.so(+0xc5a6) [0x7b38b49f45a6]
/lib/x86_64-linux-gnu/libc.so.6(+0x458d0) [0x7b38b4e458d0]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x5aace3]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(PyErr_Restore+0x9e) [0x632dde]
/home/zhongcx/sage/src/sage/libs/gap/element.cpython-313-x86_64-linux-gnu.so(+0x183f8) [0x7b385a14c3f8]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x5379c3]
/home/zhongcx/sage/src/sage/libs/gap/element.cpython-313-x86_64-linux-gnu.so(+0x3de74) [0x7b385a171e74]
/home/zhongcx/sage/src/sage/libs/gap/element.cpython-313-x86_64-linux-gnu.so(+0x3e413) [0x7b385a172413]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x5b6793]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(_PyEval_EvalFrameDefault+0x47db) [0x56d8cb]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(PyEval_EvalCode+0xcd) [0x564d3d]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x5e6385]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(_PyEval_EvalFrameDefault+0x2858) [0x56b948]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x6b4c1e]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x6fbcc3]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(_PyEval_EvalFrameDefault+0x328a) [0x56c37a]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(PyEval_EvalCode+0xcd) [0x564d3d]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x6be392]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x6bb32e]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x6cc4c4]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x6cbf45]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3() [0x6cbd85]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(Py_RunMain+0x3ea) [0x6caffa]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(Py_BytesMain+0x2d) [0x69500d]
/lib/x86_64-linux-gnu/libc.so.6(+0x2a578) [0x7b38b4e2a578]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b) [0x7b38b4e2a63b]
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/python3(_start+0x25) [0x6943b5]

GAPError is the normal return, but sometimes segfault.

@orlitzky
Copy link
Contributor

Ok, thanks. There are more CI failures but they all look unrelated.

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 14, 2025

Ok, thanks. There are more CI failures but they all look unrelated.

@orlitzky Can you see my new PR? I finally found a way to fix the element.pyx. I believe I solved it eventually

vbraun pushed a commit to vbraun/sage that referenced this pull request Aug 14, 2025
sagemathgh-40548: fix pkg_resources DeprecationWarning
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->
Fixes sagemath#38389. I think it will fix the pkg_resources Deprecation Warning
when upgrading setuptools. Because from setuptools 80.9.0, it does not
reply on the ```pkg_resources```
```
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/sage-venv-
config:4: DeprecationWarning: pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html

  __import__('pkg_resources').require('sagemath-standard==10.7rc0')

/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/sage-
ipython:4: DeprecationWarning: pkg_resources is deprecated as an API.
See https://setuptools.pypa.io/en/latest/pkg_resources.html

  __import__('pkg_resources').require('sagemath-standard==10.7rc0')
```

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [x] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#40548
Reported by: Chenxin Zhong
Reviewer(s):
@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 15, 2025

Hello @orlitzky, can you just pause it merge. Because I found in setuptools>=80 in new version remove the easy-install. It will make that when you change a file, you need to complie the whole sagelib. We may use another way to replace the easy-install. But I am not familiar with the build system. I apologized that I do not check it carefully.

Easy Install is deprecated. Do not use it. Instead use pip. If you think you need Easy Install, please reach out to the PyPA team (a ticket to pip or setuptools is fine), describing your use-case.

@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 15, 2025

Ok, thanks. There are more CI failures but they all look unrelated.

@orlitzky Now turn to this question. If we still want to update setuptools, we need to modify editable install. Because In the high er version of setuptools, easy-install is deprecated. We have to continuous this issue #34209

@cxzhong cxzhong marked this pull request as draft August 15, 2025 17:53
@orlitzky
Copy link
Contributor

Honestly you might be able to wait: #39030 will replace the build system very soon.

vbraun pushed a commit to vbraun/sage that referenced this pull request Aug 16, 2025
sagemathgh-40548: fix pkg_resources DeprecationWarning
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->
Fixes sagemath#38389. I think it will fix the pkg_resources Deprecation Warning
when upgrading setuptools. Because from setuptools 80.9.0, it does not
reply on the ```pkg_resources```
```
/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/sage-venv-
config:4: DeprecationWarning: pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html

  __import__('pkg_resources').require('sagemath-standard==10.7rc0')

/home/zhongcx/sage/local/var/lib/sage/venv-python3.13/bin/sage-
ipython:4: DeprecationWarning: pkg_resources is deprecated as an API.
See https://setuptools.pypa.io/en/latest/pkg_resources.html

  __import__('pkg_resources').require('sagemath-standard==10.7rc0')
```

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [x] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#40548
Reported by: Chenxin Zhong
Reviewer(s):
@cxzhong
Copy link
Contributor Author

cxzhong commented Aug 16, 2025

Honestly you might be able to wait: #39030 will replace the build system very soon.

yes I just wait for that and keep it draft

@cxzhong cxzhong closed this Aug 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sage shows "DeprecationWarning: pkg_resources is deprecated as an API." when __import__('pkg_resources').require('sagemath-standard==10.3')
2 participants