Render ARRAY and ITERABLE fields that are not backed by a List - #39750
Render ARRAY and ITERABLE fields that are not backed by a List#39750PDGGK wants to merge 2 commits into
Conversation
Row#toString throws for an ITERABLE field holding a plain Iterable:
Schema s = Schema.builder().addStringField("k")
.addIterableField("vals", FieldType.STRING).build();
Row.withSchema(s).attachValues("k1", () -> list.iterator()).toString();
IllegalArgumentException: value type is '...' for field type 'ITERABLE'
toPrettyFieldValueString demanded a List before iterating. An ITERABLE
field declares an Iterable, so the guard is stricter than the type it
guards, and the branch only ever iterates and counts -- both of which
Iterable supports once materialised.
Losing a whole Row's rendering is a poor trade for a stricter check
here, since this is what Row#toString calls: one unusual field takes out
logging and debugger output for every field beside it.
The guard now requires Iterable and materialises once for the size
count. A value that is neither still throws the same exception.
Two tests: a bare Iterable on an ITERABLE field, which fails on master,
and an ordinary List array as a control that passes either way. The
first also asserts the value really is not a List before rendering, so
it cannot go green for the wrong reason if the value ever starts
arriving materialised.
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
The one red check is That is the Postgres testcontainer refusing to load its logical-decoding output plugin, in |
|
assign set of reviewers |
|
Assigning reviewers: R: @ahmedabu98 for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
|
||
| @SuppressWarnings("unchecked") | ||
| List<Object> list = (List<Object>) value; | ||
| List<Object> list = Lists.newArrayList((Iterable<Object>) value); |
There was a problem hiding this comment.
nit: keep the original iterable instead of recreating the object as a list? use guava Iterables.isEmpty and Iterables.size for the lines below
There was a problem hiding this comment.
Good instinct, and I've taken it — but not with Iterables.isEmpty / Iterables.size, because those would introduce a bug here. Details, since the reason is easy to miss:
As the block stood it touched the collection three times — isEmpty(), then size(), then the for. A List absorbs that for free, which is why it was fine before. An Iterable that can only be read once does not, and that is exactly the kind of value this PR widens support to. So the copy was load-bearing rather than gratuitous.
I added a test that hands out its iterator exactly once and throws on a second attempt. With your literal suggestion applied:
testToPrettyStringRendersAnIterableThatCanOnlyBeReadOnce
java.lang.IllegalStateException: iterated more than once
1 of 13 failing, and it's that one.
What does satisfy the point is rendering in a single pass, which drops both the copy and the multi-traversal assumption:
StringBuilder sb = new StringBuilder();
sb.append("[\n");
boolean empty = true;
for (Object element : elements) {
if (!empty) {
sb.append(",\n");
}
sb.append(nextPrefix).append(toPrettyFieldValueString(elementType, element, nextPrefix));
empty = false;
}
if (empty) {
return "[]";
}
sb.append("\n").append(prefix).append("]");Output is unchanged — same ",\n" separators, and an empty collection still renders as [], which now has its own test too. Lists is no longer imported. spotlessJavaCheck, checkstyleMain and checkstyleTest are clean, and SchemaUtilsTest plus RowTest pass.
Pushed in f297bcf. Thanks for the look.
Review feedback from @ahmedabu98: keep the original iterable rather than recreating it as a list. The copy was load-bearing as the block stood, because it asked the collection for isEmpty(), then size(), then iterated it -- three traversals. A List absorbs that for free; an Iterable that can only be read once does not, and that is exactly the kind of value this PR widens support to. Doing it in a single pass gets both: no copy, and no assumption that the collection can be traversed more than once. Output is unchanged -- elements are separated by ",\n" as before, and an empty collection still renders as "[]". Two tests added. The one-shot case hands out its iterator exactly once and throws on a second attempt; with Iterables.isEmpty plus Iterables.size over the raw iterable it fails with "iterated more than once", and it is the only one of the thirteen that does.
|
Run Java_IOs_Direct PreCommit |
Fixes #39749.
Row#toStringthrows for anITERABLEfield holding a plainIterable:toPrettyFieldValueStringdemanded aListbefore iterating. AnITERABLEfield declares anIterable, so the guard was stricter than the type it guards, and the branch below only ever iterates and counts — both fine from anIterableonce materialised.This is reached from
Row#toString, so refusing to render one field takes out logging and debugger output for every field beside it. That is a poor trade for a stricter check in a renderer.The guard now requires
Iterable; a value that is neither still throws the same exception.On where this belongs
A reviewer could reasonably hold that a materialised
Rowshould always store aListforARRAY/ITERABLE, and that a non-Listreaching here means a producer is at fault —ByteBuddyUtils.transformContainerhands back aCollections2.TransformedCollectionfor aSet-typed POJO field, for instance. I think that is worth looking at separately and have not touched it.What makes this worth fixing on its own is that the reproducer needs only
Schema.builder,attachValuesandtoString()— no POJO, no registry, no pipeline — so the renderer is reachable with a merely-Iterablevalue through plain public API. And atoString()that throws is difficult to justify whatever produced the value.I should be straight about the blast radius rather than inflate it: I could not construct a case where a non-
Listreaches user code inside a running pipeline. Coder round-trips materialise, andConvert.toRows()/@Element Rowboth yieldLists. The demonstrated exposure is direct API use.Tests
Iterableon anITERABLEfieldIllegalArgumentExceptionaboveListon anARRAYfieldThe first also asserts
!(row.getValue("vals") instanceof List)before rendering, so if the value ever starts arriving materialised the test fails loudly rather than going green for the wrong reason.11/11 in
SchemaUtilsTest, and the wider*schemas*/*RowTest*suites pass.spotlessCheck,checkstyleMain,checkstyleTestclean.