Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions README/ReleaseNotes/v642/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,53 @@ resolve its symbols from the `libROOTPythonizations` Python extension module.
Replace `obj.Connect(signal, "TPyDispatcher", disp, "Dispatch()")` with
`obj.Connect(signal, callable)`.

### Equality comparisons of C++ objects follow C++ semantics

**This is a behavior change that can affect existing code.** Comparing two
Python proxies of C++ objects with `==` or `!=` now requires the C++ class to
provide the corresponding operator (if only one of `operator==` and
`operator!=` is defined, the other one is still derived from it by negation).
When there is no C++ equality operator for the operands, a `TypeError` is
raised:

```python
ROOT.gInterpreter.Declare("class MyClass {};")

a = ROOT.MyClass()
b = ROOT.MyClass()

a == b # raises TypeError in ROOT 6.42, was False before
a == a # raises TypeError in ROOT 6.42, was True before
```

Previously, such comparisons silently fell back to comparing the type and the
address of the wrapped C++ object. That fallback was misleading, since `a == b`
looked like a comparison by value even though the C++ class did not define one.
It also produced inconsistent results for classes that define only one of the
two operators: for example, `v1 == v2` for two **RooRealVar** objects with the
same value returned `True` via `RooAbsReal::operator==`, while `v1 != v2`
returned `True` as well, because `!=` fell back to comparing addresses.

If you relied on the old behavior to check whether two proxies refer to the
same C++ object, compare the addresses explicitly with
`ROOT.addressof(a) == ROOT.addressof(b)`. If a comparison by value is
meaningful for your class, define an `operator==` for it in C++, or implement
`__eq__` and `__ne__` from Python with a custom pythonization.

Two exceptions to the new rule are kept, to avoid changing behavior where the
old semantics were unambiguous:

* Proxies of the *same* type are still compared by address if at least one of
them wraps a `nullptr`, because a comparison by value is not possible then.
* Comparisons with objects that are not C++ proxies are left to Python, so
`a == 1` is still `False` and `a != 1` is still `True`.

Related to this, the undocumented `__eq__`/`__ne__` pythonization of
**TObject**, which forwarded to `TObject::IsEqual()`, is gone. Classes
inheriting from `TObject` that override `IsEqual()` need to call it explicitly,
or define a C++ `operator==`. More details can be found in the
[pythonizations documentation](https://root.cern/doc/master/group__Pythonizations.html).

## I/O

* Reading a collection without its dictionary no longer crashes when the elements hold a `std::string` or a `TString`. The emulated collection proxy relocated its elements with a raw memory copy when its buffer had to grow, which corrupts an object that points into itself, such as a `std::string` using the small string optimization; the invalid pointer was then freed when the object was destroyed. Such elements are now destroyed and reconstructed at the new location instead. This affected for instance a `std::vector<std::pair<std::string,double>>` read back without a dictionary.
Expand Down
72 changes: 39 additions & 33 deletions bindings/pyroot/cppjit/src/cpyrt/CPPInstance.cxx
Original file line number Diff line number Diff line change
Expand Up @@ -592,23 +592,6 @@ static inline PyObject* eqneq_binop(CPPClass* klass, PyObject* self,
Py_RETURN_TRUE;
}

static inline void* cast_actual(void* obj) {
void* address = ((CPPInstance*)obj)->GetObject();
if (((CPPInstance*)obj)->fFlags & CPPInstance::kIsActual)
return address;

interop::TCppScope_t klass = ((CPPClass*)Py_TYPE((PyObject*)obj))->fCppType;
interop::TCppScope_t clActual = interop::GetActualClass(klass, address);
if (clActual && clActual != klass) {
intptr_t offset = interop::GetBaseOffset(
clActual, klass, address, -1 /* down-cast */, true /* report errors */);
if (offset != -1)
address = (void*)((intptr_t)address + offset);
}

return address;
}

#define CPYRT_ORDERED_OPERATOR_STUB(op, ometh, label) \
if (!ometh) { \
PyCallable* pyfunc = \
Expand Down Expand Up @@ -659,25 +642,48 @@ static PyObject* op_richcompare(CPPInstance* self, PyObject* other, int op) {
if (result)
return result;

// default behavior: type + held pointer value defines identity; if both are
// CPPInstance objects, perform an additional autocast if need be
bool bIsEq = false;

if ((Py_TYPE(self) == Py_TYPE(other) &&
self->GetObject() == ((CPPInstance*)other)->GetObject())) {
// direct match
bIsEq = true;
} else if (CPPInstance_Check(other)) {
// try auto-cast match
void* addr1 = cast_actual(self);
void* addr2 = cast_actual(other);
bIsEq = addr1 && addr2 && (addr1 == addr2);
// for non-CPPInstance objects, let Python handle dispatch/fallback
if (!CPPInstance_Check(other)) {
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}

if ((op == Py_EQ && bIsEq) || (op == Py_NE && !bIsEq))
Py_RETURN_TRUE;
// if both proxies have the same type and at least one wraps nullptr,
// comparing nullness is unambiguous
auto ptr1 = self->GetObject();
auto ptr2 = ((CPPInstance*)other)->GetObject();
if (ptr1 == nullptr || ptr2 == nullptr) {
// Take this branch only for exact proxy type matches. Broadening this
// to inheritance-related types would be allowed in C++, but it would
// silently change previous cppyy behavior, where equality comparison
// of different-typed nullptr always resulted in `False`, straying away
// from C++.
if (Py_TYPE(self) == Py_TYPE(other)) {
bool bIsEq = ptr1 == ptr2;
if ((op == Py_EQ && bIsEq) || (op == Py_NE && !bIsEq))
Py_RETURN_TRUE;
Py_RETURN_FALSE;
}
}

Py_RETURN_FALSE;
// in the remaining cases, the semantics of the comparison are ambiguous, so
// we raise an exception
const char* opstr = op == Py_EQ ? "==" : "!=";
const char* msg =
"\n'%s' is not supported between instances of \"%s\" and \"%s\", "
"because these operands have no C++ "
"equality operator."
"\n\nIf comparing by value is meaningful for these types, define a C++ "
"`operator==` for them, or "
"implement `__eq__` and `__ne__` in Python."
"\n\nTo check whether the two proxies refer to the same C++ object, "
"compare their addresses explicitly "
"with `cppjit.addressof()`."
"\n";

PyErr_Format(PyExc_TypeError, msg, opstr, Py_TYPE(self)->tp_name,
Py_TYPE(other)->tp_name);
return NULL;
}

// ordered comparison operators
Expand Down
5 changes: 3 additions & 2 deletions bindings/pyroot/cppjit/test/test_advancedcpp.py
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,6 @@ def test10_object_identity(self):
r4 = r1.m_other
assert r3 is r4

assert r3 == r2
assert r3 is r2

r3.extra = 42
Expand Down Expand Up @@ -738,7 +737,9 @@ def test21_access_to_global_variables(self):
assert type(o) != type(cppjit.gbl.g_abstract_ptr)
cppjit.gbl.g_abstract_ptr = o
assert type(o) != type(cppjit.gbl.g_abstract_ptr)
assert cppjit.gbl.g_abstract_ptr == o
# the classes have no C++ equality operator, so the proxies have to be
# compared by address explicitly
assert cppjit.addressof(cppjit.gbl.g_abstract_ptr) == cppjit.addressof(o)
cppjit.gbl.g_abstract_ptr = cppjit.nullptr

cppjit.cppexec("std::vector<int>* gtestv1 = nullptr;")
Expand Down
8 changes: 6 additions & 2 deletions bindings/pyroot/cppjit/test/test_datatypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -1124,8 +1124,12 @@ def test19_object_and_pointer_comparisons(self):
l1 = cppjit.bind_object(0, gbl.FourVector)
assert not l1

assert c1 != l1
assert l1 != c1
# comparing null pointers of unrelated types is not supported, since
# there is no C++ equality operator for these operands
with raises(TypeError):
c1 != l1
with raises(TypeError):
l1 != c1

l2 = cppjit.bind_object(0, gbl.FourVector)
assert l1 == l2
Expand Down
6 changes: 4 additions & 2 deletions bindings/pyroot/cppjit/test/test_regression.py
Original file line number Diff line number Diff line change
Expand Up @@ -889,9 +889,11 @@ def triggerChange(self):

def changeCallback(self, b):
assert type(b) == type(self.derived)
assert b == self.derived
# the classes involved have no C++ equality operator, so the
# proxies have to be compared by address explicitly
assert cppjit.addressof(b) == cppjit.addressof(self.derived)
cast = cppjit.gbl.std.addressof[type(b)]
assert cast(b) == cast(self.derived)
assert cppjit.addressof(cast(b)) == cppjit.addressof(cast(self.derived))
self.success = True

g = Glue()
Expand Down
50 changes: 50 additions & 0 deletions bindings/pyroot/pythonizations/doc/pythonizations.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,56 @@ unconditionally. It is only available for polymorphic base
classes, which is why the `Base` class in the example has a virtual destructor.


### Equality comparisons

The `==` and `!=` operators on C++ object proxies follow C++ semantics: they
call the `operator==` or `operator!=` that the C++ class provides. If only one
of the two operators is defined, the other one is derived from it by negation.

If the class provides no equality operator at all, the comparison is **not**
silently answered by comparing the addresses of the wrapped C++ objects.
Instead, a `TypeError` is raised:

~~~{.py}
ROOT.gInterpreter.Declare("class MyClass {};")

a = ROOT.MyClass()
b = ROOT.MyClass()

a == b # raises TypeError
~~~

An address comparison would be misleading here, because `a == b` looks like a
comparison by value even though the C++ class does not define one. Note that
raising is also what protects you from Python's own fallback: if the proxy
declared the comparison unsupported, Python would silently compare object
identity, just like the `is` operator does.

There are three ways out if you hit this error:

1. Define an `operator==` for your C++ class, if comparing by value is
meaningful for it. This is the recommended solution, since the class then
behaves consistently in C++ and in Python.
2. Implement `__eq__` and `__ne__` from Python, for example with a custom
pythonization as explained in the next section.
3. If you want to know whether two proxies refer to the same C++ object,
compare their addresses explicitly:
~~~{.py}
ROOT.addressof(a) == ROOT.addressof(b)
~~~

A few special cases are worth pointing out:

* Comparing a proxy with an object that is not a C++ proxy is left to Python,
so for example `a == 1` is `False` and `a != 1` is `True`.
* Two proxies of the *same* type are still compared by address if at least
one of them wraps a `nullptr`, because comparing by value is impossible in
that case. If the types differ, a `TypeError` is raised, even though C++
would allow the comparison for related types.
* Comparing a null proxy to `None` raises a `TypeError` as well. Use the
truth value of the proxy (`if not a: ...`) to check for a null pointer, and
`a is None` to check for Python's `None`.

### Custom pythonizations for C++ user classes by example

This example shows how to use the `@pythonization` decorator to add extra
Expand Down
20 changes: 14 additions & 6 deletions bindings/pyroot/pythonizations/test/tobject.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

# Helpers
def create_tlist(self):
l = ROOT.TList()

Check failure on line 20 in bindings/pyroot/pythonizations/test/tobject.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E741)

bindings/pyroot/pythonizations/test/tobject.py:20:9: E741 Ambiguous variable name: `l`
for _ in range(self.num_elems):
o = ROOT.TObject()
# Prevent immediate deletion of C++ TObjects
Expand All @@ -28,7 +28,7 @@

# Tests
def test_contains(self):
l = self.create_tlist()

Check failure on line 31 in bindings/pyroot/pythonizations/test/tobject.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E741)

bindings/pyroot/pythonizations/test/tobject.py:31:9: E741 Ambiguous variable name: `l`

for elem in l:
self.assertTrue(elem in l)
Expand All @@ -44,8 +44,10 @@
"""
Test for the comparison operators of TObject and subclasses:
__eq__, __ne__, __lt__, __le__, __gt__, __ge__.
Such pythonizations rely on TObject::IsEqual and TObject::Compare,
which can be reimplemented in subclasses.
The ordering pythonizations rely on TObject::Compare, which can be
reimplemented in subclasses. There is no __eq__/__ne__ pythonization:
equality follows C++ semantics, so comparing objects of a class without
a C++ equality operator raises a TypeError.
"""

num_elems = 3
Expand All @@ -54,26 +56,32 @@
def test_eq(self):
o = ROOT.TObject()

# TObject::IsEqual compares internal address
self.assertTrue(o == o)
# TObject has no C++ operator==, so the comparison is not supported.
# To compare by address, TObject::IsEqual can be used explicitly.
with self.assertRaises(TypeError):
o == o
self.assertTrue(o.IsEqual(o))

# Test comparison with no TObject
self.assertFalse(o == 1)

# Test comparison with None
self.assertFalse(o == None)

Check failure on line 69 in bindings/pyroot/pythonizations/test/tobject.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E711)

bindings/pyroot/pythonizations/test/tobject.py:69:31: E711 Comparison to `None` should be `cond is None` help: Replace with `cond is None`

def test_ne(self):
o = ROOT.TObject()

# TObject::IsEqual compares internal address
self.assertFalse(o != o)
# TObject has no C++ operator!=, so the comparison is not supported.
# To compare by address, TObject::IsEqual can be used explicitly.
with self.assertRaises(TypeError):
o != o
self.assertFalse(not o.IsEqual(o))

# Test comparison with no TObject
self.assertTrue(o != 1)

# Test comparison with None
self.assertTrue(o != None)

Check failure on line 84 in bindings/pyroot/pythonizations/test/tobject.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E711)

bindings/pyroot/pythonizations/test/tobject.py:84:30: E711 Comparison to `None` should be `cond is not None` help: Replace with `cond is not None`

def test_nullptr_eq_none_raises(self):
import cppyy
Expand All @@ -86,10 +94,10 @@
# ROOT versions, which would be a silent behavior change.
# See https://github.com/root-project/root/issues/20283
with self.assertRaises(TypeError):
_ = x == None

Check failure on line 97 in bindings/pyroot/pythonizations/test/tobject.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E711)

bindings/pyroot/pythonizations/test/tobject.py:97:22: E711 Comparison to `None` should be `cond is None` help: Replace with `cond is None`

with self.assertRaises(TypeError):
_ = x != None

Check failure on line 100 in bindings/pyroot/pythonizations/test/tobject.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E711)

bindings/pyroot/pythonizations/test/tobject.py:100:22: E711 Comparison to `None` should be `cond is not None` help: Replace with `cond is not None`

def test_lt(self):
a = TUrl("a")
Expand Down
6 changes: 5 additions & 1 deletion roottest/python/regression/PyROOT_regressiontests.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
ROOT-specific tests and tests without upstream equivalents.
"""

import platform

Check failure on line 16 in roottest/python/regression/PyROOT_regressiontests.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (F401)

roottest/python/regression/PyROOT_regressiontests.py:16:8: F401 `platform` imported but unused help: Remove unused import: `platform`
import sys, os, unittest

Check failure on line 17 in roottest/python/regression/PyROOT_regressiontests.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E401)

roottest/python/regression/PyROOT_regressiontests.py:17:1: E401 Multiple imports on one line help: Split imports

Check failure on line 17 in roottest/python/regression/PyROOT_regressiontests.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (I001)

roottest/python/regression/PyROOT_regressiontests.py:16:1: I001 Import block is un-sorted or un-formatted help: Organize imports
sys.path.append(os.path.dirname(os.path.dirname(__file__)))

if not os.path.exists('ScottCppyy.C'):
Expand All @@ -29,7 +29,7 @@

original_preload = os.environ.get('LD_PRELOAD', None)

import ROOT

Check failure on line 32 in roottest/python/regression/PyROOT_regressiontests.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (E402)

roottest/python/regression/PyROOT_regressiontests.py:32:1: E402 Module level import not at top of file help: Move module level imports to top of file
ROOT.PyConfig.IgnoreCommandLineOptions = False
from ROOT import gROOT, gInterpreter
from ROOT import TClass, TObject, TFile
Expand Down Expand Up @@ -245,7 +245,11 @@

b = a.begin()
e = a.end()
self.assertNotEqual( a, e )
self.assertNotEqual( b, e )

# the AttributeList itself has no equality operator with an iterator
with self.assertRaises(TypeError):
a == e

b.__preinc__()
self.assertEqual( b, e )
Expand Down
Loading