Skip to content

release-8.5: clarify IMPORT INTO format detection by platform and version - #23490

Open
alastori wants to merge 4 commits into
pingcap:release-8.5from
alastori:docs/import-into-format-detection-8.5
Open

release-8.5: clarify IMPORT INTO format detection by platform and version#23490
alastori wants to merge 4 commits into
pingcap:release-8.5from
alastori:docs/import-into-format-detection-8.5

Conversation

@alastori

@alastori alastori commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

What is changed, added or deleted? (Required)

This PR aligns the IMPORT INTO format documentation on release-8.5 with the behavior released in TiDB v8.5.7 and the current TiDB Cloud behavior:

  • Explains that an explicit FORMAT clause overrides file-extension detection.
  • Qualifies automatic format detection as available in TiDB Self-Managed starting from v8.5.7.
  • Preserves the v8.5.6-and-earlier behavior in which omitting FORMAT uses CSV.
  • Uses platform-specific content so the TiDB Cloud documentation does not expose a Self-Managed patch-version qualifier.
  • Documents case-insensitive detection for .csv, .sql, and .parquet files in the applicable versions and platform.
  • Lists the supported compression suffixes ignored during detection.
  • Clarifies that a missing or unrecognized extension on external storage falls back to CSV, while TiDB local paths require a supported final suffix.
  • Explains that wildcard imports use one format for all matched files and avoids claiming that mixed formats fail during precheck.

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

  • master (the latest development version)
  • v8.5 (TiDB 8.5 versions)
  • v8.4 (TiDB 8.4 versions)
  • v8.3 (TiDB 8.3 versions)
  • v8.2 (TiDB 8.2 versions)
  • v8.1 (TiDB 8.1 versions)
  • v7.5 (TiDB 7.5 versions)
  • v7.1 (TiDB 7.1 versions)
  • v6.5 (TiDB 6.5 versions)

What is the related PR or file link(s)?

Validation

  • ./scripts/markdownlint sql-statements/sql-statement-import-into.md
  • python3 scripts/file-format-lint.py sql-statements/sql-statement-import-into.md
  • python3 scripts/check-manual-line-breaks.py sql-statements/sql-statement-import-into.md
  • git diff --check
  • Verified that the platform filters retain the v8.5.7 qualifier and earlier-version fallback for TiDB Self-Managed, while TiDB Cloud retains the current automatic-detection wording.
  • Local documentation review gate: APPROVED, with no findings.
  • Ran go test -tags=intest,deadlock ./pkg/executor/importer -run 'Test(SupportedSuffixForServerDisk|ParseFileType)$' -count=1 against the TiDB v8.5.7 tag.
  • Direct smoke test against TiDB v8.5.7 and an S3-compatible MinIO endpoint. This validates the TiDB Self-Managed engine behavior for case-insensitive CSV, SQL, and Parquet detection; .gz, .gzip, .zstd, .zst, and .snappy suffixes; missing and unrecognized extension fallback; explicit FORMAT override; and homogeneous wildcard imports.
  • A mixed .csv and .sql wildcard created an import job and failed while parsing the SQL file as CSV. It did not fail during precheck, and the target table remained empty.
  • A live TiDB Cloud cluster was not exercised in this smoke test.

AI agent involvement

  • The changes in this PR were primarily made by an AI agent on behalf of the PR author.

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

Summary by CodeRabbit

  • Documentation
    • Clarified that format detection is automatic and case-insensitive when FORMAT is omitted.
    • Documented support for recognized compression suffixes in local file paths, regardless of letter case.
    • Explained compression-suffix handling, version-specific behavior, and CSV fallback.
    • Added local-path validation guidance for imports.
    • Noted that files matched by wildcards must share the same format and compatible naming conventions.

@ti-chi-bot

ti-chi-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 11, 2026
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 89a9b24b-49fc-4193-a132-b2ed7d5677c0

📥 Commits

Reviewing files that changed from the base of the PR and between b51acd5 and 5bf118e.

📒 Files selected for processing (1)
  • sql-statements/sql-statement-import-into.md

📝 Walkthrough

Walkthrough

The IMPORT INTO documentation now explains case-insensitive format detection, compression suffix handling, local-path validation, version-specific behavior, CSV fallback, and wildcard format requirements.

Changes

IMPORT INTO format documentation

Layer / File(s) Summary
Document FORMAT detection behavior
sql-statements/sql-statement-import-into.md
The documentation describes supported extensions, compression handling, case-insensitive detection, local-path validation, version differences, CSV fallback, and the requirement that wildcard-matched files use one format.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Merge Risk: 🔵 Low · up to 5bf11

The documentation changes format detection guidance by platform and version, but the TiDB Cloud wording may conflict with its documented CSV default when FORMAT is omitted. Confirming the supported Cloud behavior and release scope is needed to avoid misleading users about SQL and Parquet imports.

Suggested labels: contribution

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the release branch and the main documentation change for IMPORT INTO format detection.
Description check ✅ Passed The description covers the required change summary, affected version, references, AI checklist, change categories, and detailed validation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot

ti-chi-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign csuzhangxc for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added missing-translation-status This PR does not have translation status info. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Aug 11, 2026
@alastori

Copy link
Copy Markdown
Collaborator Author

@JQWong7, could you review this documentation update for the IMPORT INTO automatic format-detection behavior implemented in pingcap/tidb#59543?

I smoke-tested the release-8.5 backport on TiDB v8.5.7, including explicit FORMAT, case-insensitive file and compression extensions, CSV fallback, and wildcard imports. The mixed-format case failed during the import rather than during precheck, so this draft avoids that earlier documentation claim.

@ti-chi-bot ti-chi-bot Bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Aug 11, 2026
@alastori alastori changed the title release-8.5: document IMPORT INTO automatic format detection release-8.5: clarify IMPORT INTO format detection by platform and version Aug 11, 2026
@alastori
alastori force-pushed the docs/import-into-format-detection-8.5 branch from d5882ac to 3f69993 Compare August 14, 2026 05:30
@alastori

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 03a36f93-0a68-4d35-b10e-e14e8d1e854f

📥 Commits

Reviewing files that changed from the base of the PR and between b51acd5 and 3f69993.

📒 Files selected for processing (1)
  • sql-statements/sql-statement-import-into.md

Comment thread sql-statements/sql-statement-import-into.md Outdated
Comment thread sql-statements/sql-statement-import-into.md Outdated
@alastori

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
sql-statements/sql-statement-import-into.md (1)

135-135: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Limit the explicit-FORMAT claim to format selection.

Line 135 says that FORMAT works regardless of the file extension, but Line 141 says TiDB rejects unsupported final suffixes for local paths. Clarify that explicit FORMAT controls parsing and does not bypass local-path suffix validation.

Committable replacement
-The `IMPORT INTO` statement supports three data file formats: `CSV`, `SQL`, and `PARQUET`. When you specify the `FORMAT` clause, TiDB uses that format regardless of the file extension.
+The `IMPORT INTO` statement supports three data file formats: `CSV`, `SQL`, and `PARQUET`. When you specify the `FORMAT` clause, TiDB uses that format for parsing instead of detecting it from the file extension. For TiDB local file paths, the final suffix validation described above still applies.

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 1ffaec93-6026-49d1-8112-d7de51da71bc

📥 Commits

Reviewing files that changed from the base of the PR and between b51acd5 and 944e270.

📒 Files selected for processing (1)
  • sql-statements/sql-statement-import-into.md

@alastori

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@alastori
alastori marked this pull request as ready for review August 14, 2026 06:01
@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 14, 2026
@CLAassistant

CLAassistant commented Aug 14, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@alastori

Copy link
Copy Markdown
Collaborator Author

@JQWong7 PTAL. You implemented the underlying automatic file type detection in pingcap/tidb#59543, so your review of the platform-specific and version-specific behavior documented here would be especially helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

missing-translation-status This PR does not have translation status info. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants