Skip to content

[FLINK-39789] AvroSchemaConverter updates for local-timestamp-* and timestamp-nanos logical types#28274

Open
Shekharrajak wants to merge 4 commits into
apache:masterfrom
Shekharrajak:FLINK-39789-avro-local-timestamp-rel21
Open

[FLINK-39789] AvroSchemaConverter updates for local-timestamp-* and timestamp-nanos logical types#28274
Shekharrajak wants to merge 4 commits into
apache:masterfrom
Shekharrajak:FLINK-39789-avro-local-timestamp-rel21

Conversation

@Shekharrajak
Copy link
Copy Markdown
Contributor

@Shekharrajak Shekharrajak commented May 29, 2026

Fix FLINK-39789.

What is the purpose of the change

AvroSchemaConverter.convertToDataType(String) and convertToTypeInfo(String) — the only public Avro→Flink type-mapping entry points — silently dropped the logical type for local-timestamp-millis and local-timestamp-micros and returned BIGINT / Long, producing a Flink RowType that is incompatible with TimestampData at runtime.

The internal two-arg overloads contained branches for local-timestamp-*, but only on the legacyTimestampMapping=false arm. The single-arg public APIs hard-code legacyTimestampMapping=true, so those branches were unreachable from any public API.

Brief change log

Verifying this change

  • AvroSchemaConverter.convertToDataType(Schema, boolean) — hoist localTimestampMillis / localTimestampMicros checks above the legacyMapping switch so they are honoured in both modes.
  • AvroSchemaConverter.convertToTypeInfo(Schema, boolean) — identical hoist, mapping to Types.LOCAL_DATE_TIME.
  • Add AvroSchemaConverterLocalTimestampBehaviorTest — 13 tests covering the single-arg public API, the two-arg overloads in both modes, the convertToTypeInfo mirror, the JIRA's exact runtime ClassCastException reproducer, and a round-trip sanity check.
  • Update AvroSchemaConverterTest.validateLegacyTimestampsSchema(...) helpers — they previously hard-coded Types.LONG / BIGINT for local_timestamp_* (asserting the bug); now assert the spec-correct Types.LOCAL_DATE_TIME / TIMESTAMP(p).

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no) no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)no
  • The serializers: (yes / no / don't know)no
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)no
  • The S3 file system connector: (yes / no / don't know)no

Documentation

  • Does this pull request introduce a new feature? (yes / no)no
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)no

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

@Shekharrajak Shekharrajak marked this pull request as draft May 29, 2026 03:49
@flinkbot
Copy link
Copy Markdown
Collaborator

flinkbot commented May 29, 2026

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@Shekharrajak Shekharrajak force-pushed the FLINK-39789-avro-local-timestamp-rel21 branch from be77e45 to dc5e6de Compare May 29, 2026 04:10
@Shekharrajak Shekharrajak marked this pull request as ready for review May 29, 2026 14:25
@Shekharrajak Shekharrajak changed the title Flink 39789 avro local timestamp rel21 AvroSchemaConverter.convertToDataType(String) updates for local-timestamp-* and timestamp-nanos logical types May 29, 2026
@Shekharrajak Shekharrajak changed the title AvroSchemaConverter.convertToDataType(String) updates for local-timestamp-* and timestamp-nanos logical types AvroSchemaConverter updates for local-timestamp-* and timestamp-nanos logical types May 29, 2026
@Shekharrajak Shekharrajak changed the title AvroSchemaConverter updates for local-timestamp-* and timestamp-nanos logical types [FLINK-39789] AvroSchemaConverter updates for local-timestamp-* and timestamp-nanos logical types May 29, 2026
@Shekharrajak
Copy link
Copy Markdown
Contributor Author

@flinkbot run azure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants