Build: Bump pyspark[connect] from 4.0.1 to 4.2.0

[dependabot]

Bumps pyspark[connect] from 4.0.1 to 4.2.0.

Commits

• 32f7299 Preparing Spark release v4.2.0-rc6
• 7a25c75 [SPARK-56972][SS][FOLLOWUP] Reject sink evolution combined with async progres...
• 44daf01 [SPARK-58024][PYTHON] Convert Arrow struct and map columns to Python rows in ...
• d0856c7 [SPARK-58019][PYTHON] Convert Arrow list columns to Python rows in bulk
• e79fd9b [MINOR][UI][4.2] Encode query parameters when generating log page scripts
• fc46f2a [SPARK-57962][PYTHON] Guard against path traversal in install_spark tar extra...
• 380776d [MINOR][UI] Escape HTML when rendering error messages in the Web UI tables
• c40f044 [SQL] Allow more control over UDT creation during input loading
• 5a347f9 [SPARK-58042][CONNECT] Validate the UDT jvm_class is a UserDefinedType before...
• e3c6298 [SPARK][SQL] LDAP Thriftserver improvement
• Additional commits viewable in compare view

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

[ebyhr]

=========================== short test summary info ============================
ERROR tests/integration/test_add_files.py::test_add_files_to_unpartitioned_table[format_version=1] - FutureWarning: PySpark does not yet fully support pandas >= 3.0.0. Some features may not work correctly. It is recommended to use pandas < 3.0.0 for now.
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
====================== 3922 deselected, 1 error in 19.76s ======================

[kevinjqliu]

i think we can skip this for now

[Fokko]

I think we should bump Spark altogether to 4.1. There is Spark 4.1 support for Iceberg Java 1.11

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

[github-actions]

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

[dependabot]

OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting @dependabot ignore this major version or @dependabot ignore this minor version. You can also ignore all major, minor, or patch releases for a dependency by adding an ignore condition with the desired update_types to your config file.

If you change your mind, just re-open this PR and I'll resolve any conflicts on it.