Skip to content

feat(arrow/array): Add Builder.Truncate() method#981

Closed
serramatutu wants to merge 9 commits into
apache:mainfrom
serramatutu:serramatutu/Truncate
Closed

feat(arrow/array): Add Builder.Truncate() method#981
serramatutu wants to merge 9 commits into
apache:mainfrom
serramatutu:serramatutu/Truncate

Conversation

@serramatutu

@serramatutu serramatutu commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

You can review commit by commit.

Rationale for this change

While working on this change, I ran into an issue: when a struct builder runs into an error while decoding one of the columns from JSON, there is no way to "un-append" rows from its sibling rows.

The two options are to either (1) AppendNull() to all siblings and return the error, which would leave a bad row behind, or (2) eagerly return err and leave the builder in a broken state.

for i := 0; i < b.schema.NumFields(); i++ {
	val, ok := keylist[b.schema.Field(i).Name]
	if !ok {
		b.fields[i].AppendNull()
		continue
	}

	valDec := json.NewDecoder(bytes.NewReader(val))
	valDec.UseNumber()
	if err := b.fields[i].UnmarshalOne(valDec); err != nil {
		// TODO: what to do here? We might already have appended to previous siblings (i-1, i-2, ...)
	}
}

What I want to do instead is:

if err := b.fields[i].UnmarshalOne(valDec); err != nil {
    for j := 0; j < i; j++ {
        b.fields[j].Truncate(b.fields[j].Len() - 1)
    }
    return err
}

What changes are included in this PR?

Add Truncate(n int) to the Builder interface. This new method will keep the underlying buffers but truncate the arrays to at most length n. It keeps Len() and NullN() sound. Calling b.Truncate(b.Len() - 1) is the equivalent of "un-appending" one element.

Are these changes tested?

Yes, I added a TestBuilderTruncate that checks that Truncate works for all data types.

It's worth noting I wanted to call ValidateFull() after truncate, but it seems like REE arrays have an issue with ValidateFull() that precedes this PR.

Are there any user-facing changes?

Yes, this adds a new method to the public API of Builder

@serramatutu serramatutu changed the title Add Builder.Truncate() method feat(arrow/array): Add Builder.Truncate() method Jul 22, 2026
@serramatutu

Copy link
Copy Markdown
Contributor Author

After talking with Matt I decided to close this and do something else for that PR instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant