gh-146031: Allow keeping specialization enabled when specifying eval frame function

[DinoV]

An eval frame evaluator doesn't necessarily want to disable specialization of Python -> Python calls... The custom eval frame can either hook the specializing functions its self or they can specify vectorcall on a function object which prevents the nested dispatch.

This adds an argument to _PyInterpreterState_SetEvalFrameFunc which allows the specification of whether or not specialization should be allowed. Setting it to 1 will allow the specializer to optimize Python -> Python calls. It's up to the eval frame replacement to make sure that this will work for them.

• Issue: Specialization never occurs with custom eval frame evaluator #146031

---

📚 Documentation preview 📚: https://cpython-previews--146032.org.readthedocs.build/

[markshannon]

With the flag set, some calls to any given Python function will go through the custom function and some won't.
There will be no obvious pattern, as it depends whether the caller happens to be written in Python or in C.
Is this what you want?

As soon as any function is run in the interpreter, every function called by it (transitively) could be invisible to the custom frame evaluator.

[DinoV]

With the flag set, some calls to any given Python function will go through the custom function and some won't. There will be no obvious pattern, as it depends whether the caller happens to be written in Python or in C. Is this what you want?

As soon as any function is run in the interpreter, every function called by it (transitively) could be invisible to the custom frame evaluator.

Yep, it's then up to the replacement interpreter to decide what it wants to do here. If they're just a wrapper than they probably don't want to set the flag. If they're actually interpreting the byte code than they are responsible for the implementation of the inlined calls and can dispatch to them in their own inlined interpreter.

[markshannon]

This seems reasonable.

The performance impact should be zero for CPython (as the new flag is only checked if PEP 523 is active).
If you add a new function to set the flag, the PEP 523 interface will be unchanged.

[markshannon]

I don't think we want to change this API again. It break PyTorch every time we do so.

Maybe add another function to set the flag.

[markshannon]

Can you use your IS_PEP523_HOOKED macro here?

[ashm-dev]

It is better to use the .count method of arrays here. I.e. self.assertEqual(actual_calls.count('func'), 0)

[markshannon]

The docs need updating.
Is it worth adding a "what's new" entry, so that the PyTorch developers (and anyone else using PEP 523) are made aware of this?

Other than that, the code looks good.

[markshannon]

I think you forgot to revert this and the text below.

Don't forget to add the new _PyInterpreterState_SetEvalFrameAllowSpecialization function as well

[DinoV]

The docs need updating. Is it worth adding a "what's new" entry, so that the PyTorch developers (and anyone else using PEP 523) are made aware of this?

Other than that, the code looks good.

I'm not sure it deserves a top-level what's new, there is a news entry but I don't think it quite bubbles up to the level of major new feature. I can at least ping the PyTorch people.