PEP 669 compatibility issues with PEP 523

[williamwen42]

I recently ran into PEP 669 and PEP 523 compatibility issues when developing torch.compile (pytorch/pytorch#158164).

We use PEP 523 in order to trace through Python frame objects to extract PyTorch ops. Using PEP 523 enables to automatically trace through frames without requiring extensive explicit monkey patching.

With the introduction of PEP 669, I discovered that functions calls from sys.monitoring are also going through our custom eval frame, which we don't trace very well. The workaround for the above issue was to detect any function registered to sys.monitoring and to skip tracing into them whenever they are called, even if they are called elsewhere (pytorch/pytorch#158171). However, this solution is a fairly clunky solution (calling a function registered to sys.monitoring elsewhere will not be compiled) and could add additional overhead to our compiler.

The heart of our issue is that we do not want sys.monitoring frames to go through our eval frame handler. I had some ideas on what could be done on the CPython side for a cleaner solution to our problem:

• Mark a frame if it is an instrumentation call (i.e. a call made through this line

  cpython/Python/instrumentation.c

  Line 985 in 59e2330

  PyObject *res = _PyObject_VectorcallTstate(tstate, instrument, args, nargsf, NULL);

  ) - our eval frame handler can then skip this frame if marked.
• Include an option for sys.monitoring frames to bypass the custom eval frame (and just default evaluate) - our eval frame handler will never see instrumentation frames.
• I thought about having a way to quickly temporarily disable all sys.monitoring callables, but it was hard to reason about how our eval frame handler would use this, especially because we do want to enable sys.monitoring callables when we eventually call _PyEval_EvalFrameDefault.

cc @markshannon @anijain2305

[markshannon]

PEP 523 gives you full control over execution.
If you don't want to trace sys.monitoring callbacks, then don't.

If the tracing field of the thread state is non-zero, then you are in a callback.

[williamwen42]

Ah, I believe tracing is exactly what I'm looking for. Thanks!