Skip to content

Commit 39e47c3

Browse files
committed
merge and reword the whole junk mechanism
1 parent f6e54c2 commit 39e47c3

1 file changed

Lines changed: 12 additions & 23 deletions

File tree

Doc/library/difflib.rst

Lines changed: 12 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -40,29 +40,18 @@ diffs. For comparing directories and files, see also, the :mod:`filecmp` module.
4040
complicated way on how many elements the sequences have in common; best case
4141
time is linear.
4242

43-
**User-defined junk predicate**: :class:`SequenceMatcher` accepts an *isjunk*
44-
parameter to let users determine which lines should be ignored while computing
45-
the diff.
46-
47-
.. note::
48-
49-
This predicate only ever inspects the second sequence for junk. That has
50-
an unexpected consequence: swapping the inputs can produce a different
51-
diff than simply reversing the output, sometimes very large.
52-
53-
**Automatic junk heuristic:** :class:`SequenceMatcher` supports a heuristic that
54-
automatically treats certain sequence items as junk. The heuristic counts how many
55-
times each individual item appears in the sequence. If an item's duplicates (after
56-
the first one) account for more than 1% of the sequence and the sequence is at least
57-
200 items long, this item is marked as "popular" and is treated as junk for
58-
the purpose of sequence matching. This heuristic can be turned off by setting
59-
the ``autojunk`` argument to ``False`` when creating the :class:`SequenceMatcher`.
60-
61-
.. note::
62-
63-
This heuristic only ever inspects the second sequence for junk. That has
64-
an unexpected consequence: swapping the inputs can produce a different
65-
diff than simply reversing the output, sometimes very large.
43+
**Junk**: :class:`SequenceMatcher` accepts an ``isjunk`` predicate and an
44+
``autojunk`` flag. Items that are considered as junk will not be considered
45+
to find similar content blocks. This can produce better results for humans
46+
(typically breaking on whitespace) and faster (because it reduces the number
47+
of possible combinations). But it can also cause pathological cases where
48+
too many items considered junk cause an unexpectedly large (but correct)
49+
diff result.
50+
You should consider tuning them or turning them off depending on your data.
51+
Moreover, only the second sequence is inspected for junk. This causes the diff
52+
output to not be symmetrical.
53+
When ``autojunk=True``, it will consider as junk the items that account for more
54+
than 1% of the sequence, if it is at least 200 items long.
6655

6756
.. versionchanged:: 3.2
6857
Added the *autojunk* parameter.

0 commit comments

Comments
 (0)