From 10cd9433a92ebf395fffe0f2eaca4e9b648319f1 Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Wed, 12 Aug 2026 01:09:29 -0400 Subject: [PATCH 1/8] docs: update Premium manual import mapping --- .../naming-conventions-for-data-import.md | 8 +++- .../premium/import-csv-files-premium.md | 44 +++++++++---------- tidb-cloud/premium/import-from-s3-premium.md | 6 ++- 3 files changed, 30 insertions(+), 28 deletions(-) diff --git a/tidb-cloud/naming-conventions-for-data-import.md b/tidb-cloud/naming-conventions-for-data-import.md index fe8a355f9afeb..1ab47079e62f4 100644 --- a/tidb-cloud/naming-conventions-for-data-import.md +++ b/tidb-cloud/naming-conventions-for-data-import.md @@ -119,9 +119,13 @@ If the SQL file is exported through TiDB Dumpling with the default configuration ## File pattern -If the source data file of CSV or Parquet does not conform to the naming convention, you can manually map the source data file to the target table using a file name pattern. This feature does not support Aurora Snapshot and SQL data files. +If a CSV or Parquet source file does not conform to the naming convention, you can manually map the source file to a target table using a file name pattern. Manual file-pattern mapping does not support SQL data files. -In the import wizard, on the **Destination Mapping** step, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern that supports the `*` and `?` wildcards. +In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern relative to the source URI and supports the `*` and `?` wildcards. Add one mapping for each target table. + +> **Note:** +> +> For Parquet files exported from an Aurora Snapshot, manual mapping applies only the source patterns that you configure. It does not infer a complete snapshot mapping or create the target schema. Create the target databases and tables before the import, add a mapping for each target table, and verify that the pre-check scans the expected number of data files and maps each source pattern to the intended target table. - For CSV files, see [Step 4. Import CSV files to TiDB Cloud](/tidb-cloud/import-csv-files.md#step-4-import-csv-files-to-tidb-cloud). - For Parquet files, see [Step 4. Import Parquet files to TiDB Cloud](/tidb-cloud/import-parquet-files.md#step-4-import-parquet-files-to-tidb-cloud). diff --git a/tidb-cloud/premium/import-csv-files-premium.md b/tidb-cloud/premium/import-csv-files-premium.md index 8f08d8ff3a75b..11daa2a47e443 100644 --- a/tidb-cloud/premium/import-csv-files-premium.md +++ b/tidb-cloud/premium/import-csv-files-premium.md @@ -116,7 +116,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 4. Click **Next**. -5. In the **Source Files Mapping** section, {{{ .premium }}} scans the bucket and proposes mappings between the source files and destination tables. +5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. When a directory is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. @@ -124,21 +124,19 @@ To import the CSV files to {{{ .premium }}}, take the following steps: > > When a single file is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is not displayed, and {{{ .premium }}} automatically populates the **Source** field with the file name. In this case, you only need to select the target database and table for data import. - - Leave automatic mapping enabled to apply the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. Keep **CSV** selected as the data format. + - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. - - **Advanced options**: expand the panel to view the `Ignore compatibility checks (advanced)` toggle. Leave it disabled unless you intentionally want to bypass schema compatibility validation. + - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: - - > **Note:** - > - > Manual mapping is coming soon. When the toggle becomes available, clear the automatic mapping option and configure the mapping manually: - > - > - **Source**: enter a filename pattern such as `TableName.01.csv`. Wildcards `*` and `?` are supported (for example, `my-data*.csv`). - > - **Target Database** and **Target Table**: choose the destination objects for the matched files. + - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `?` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`. + - **Target Database** and **Target Table**: enter the destination database and table for the matched files. + - To add another mapping, click **+**. + +6. Click **Next** to run the pre-check. Review the scan results and verify the source files and target tables. -6. {{{ .premium }}} automatically scans the source path. Review the scan results, check the data files found and corresponding target tables, and then click **Start Import**. +7. Click **Start Import**. -7. When the import progress shows **Completed**, check the imported tables. +8. When the import progress shows **Completed**, check the imported tables. @@ -168,7 +166,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 4. Click **Next**. -5. In the **Source Files Mapping** section, {{{ .premium }}} scans the bucket and proposes mappings between the source files and destination tables. +5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. When a directory is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. @@ -176,21 +174,19 @@ To import the CSV files to {{{ .premium }}}, take the following steps: > > When a single file is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is not displayed, and {{{ .premium }}} automatically populates the **Source** field with the file name. In this case, you only need to select the target database and table for data import. - - Leave automatic mapping enabled to apply the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. Keep **CSV** selected as the data format. + - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. - - **Advanced options**: expand the panel to view the `Ignore compatibility checks (advanced)` toggle. Leave it disabled unless you intentionally want to bypass schema compatibility validation. + - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: - - > **Note:** - > - > Manual mapping is coming soon. When the toggle becomes available, clear the automatic mapping option and configure the mapping manually: - > - > - **Source**: enter a filename pattern such as `TableName.01.csv`. Wildcards `*` and `?` are supported (for example, `my-data*.csv`). - > - **Target Database** and **Target Table**: choose the destination objects for the matched files. + - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `?` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`. + - **Target Database** and **Target Table**: enter the destination database and table for the matched files. + - To add another mapping, click **+**. + +6. Click **Next** to run the pre-check. Review the scan results and verify the source files and target tables. -6. {{{ .premium }}} automatically scans the source path. Review the scan results, check the data files found and corresponding target tables, and then click **Start Import**. +7. Click **Start Import**. -7. When the import progress shows **Completed**, check the imported tables. +8. When the import progress shows **Completed**, check the imported tables. diff --git a/tidb-cloud/premium/import-from-s3-premium.md b/tidb-cloud/premium/import-from-s3-premium.md index 3c0b5d873175d..03319b7bad303 100644 --- a/tidb-cloud/premium/import-from-s3-premium.md +++ b/tidb-cloud/premium/import-from-s3-premium.md @@ -16,7 +16,7 @@ This document describes how to import CSV files from Amazon Simple Storage Servi - To ensure data consistency, {{{ .premium }}} allows importing CSV files into empty tables only. If the target table already contains data, import into a staging table and then copy the rows using the `INSERT ... SELECT` statement. - During the public preview, the user interface currently supports Amazon S3 as the only storage provider. Support for additional providers will be added in future releases. -- Each import job maps a single source pattern to one destination table. +- Each source pattern maps to one destination table. You can add multiple mappings to an import job. ## Step 1. Prepare the CSV files @@ -55,7 +55,9 @@ The wizard includes a helper link labeled **Click here to create a new one with - Click **Test Bucket Access** to validate connectivity. 4. Click **Next** and provide the TiDB SQL username and password for the import job. Optionally, test the connection. -5. Review the automatically generated source-to-target mapping. Disable automatic mapping if you need to define custom patterns and destination tables. +5. Configure the source-to-target mapping: + - To use automatic mapping, leave **Use TiDB file naming conventions for automatic mapping** selected. + - To configure mappings manually, deselect the automatic mapping option. For each target table, enter a source file pattern relative to the **Source Files URI**, and then enter the target database and table. The source pattern supports `*` and `?` wildcards. To add another mapping, click **+**. 6. Click **Next** to run the pre-check. Resolve any warnings about missing files or incompatible schemas. 7. Click **Start Import** to launch the job group. 8. Monitor the job statuses until they show **Completed**, then verify the imported data in TiDB Cloud. From a69e4ace1d963d1afebbb95b1c8ade945b6ee7cc Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Wed, 12 Aug 2026 01:32:25 -0400 Subject: [PATCH 2/8] docs: scope Premium import mapping guidance --- .../naming-conventions-for-data-import.md | 17 ++++++++++++++++- tidb-cloud/premium/import-csv-files-premium.md | 8 ++++---- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/tidb-cloud/naming-conventions-for-data-import.md b/tidb-cloud/naming-conventions-for-data-import.md index 1ab47079e62f4..609a0c81bcaee 100644 --- a/tidb-cloud/naming-conventions-for-data-import.md +++ b/tidb-cloud/naming-conventions-for-data-import.md @@ -119,13 +119,28 @@ If the SQL file is exported through TiDB Dumpling with the default configuration ## File pattern -If a CSV or Parquet source file does not conform to the naming convention, you can manually map the source file to a target table using a file name pattern. Manual file-pattern mapping does not support SQL data files. +If a CSV or Parquet source file does not conform to the naming convention, you can manually map the source file to a target table using a file name pattern. + + + +Manual file-pattern mapping does not support Aurora Snapshot or SQL data files. + + + + +Manual file-pattern mapping does not support SQL data files. + + In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern relative to the source URI and supports the `*` and `?` wildcards. Add one mapping for each target table. + + > **Note:** > > For Parquet files exported from an Aurora Snapshot, manual mapping applies only the source patterns that you configure. It does not infer a complete snapshot mapping or create the target schema. Create the target databases and tables before the import, add a mapping for each target table, and verify that the pre-check scans the expected number of data files and maps each source pattern to the intended target table. + + - For CSV files, see [Step 4. Import CSV files to TiDB Cloud](/tidb-cloud/import-csv-files.md#step-4-import-csv-files-to-tidb-cloud). - For Parquet files, see [Step 4. Import Parquet files to TiDB Cloud](/tidb-cloud/import-parquet-files.md#step-4-import-parquet-files-to-tidb-cloud). diff --git a/tidb-cloud/premium/import-csv-files-premium.md b/tidb-cloud/premium/import-csv-files-premium.md index 11daa2a47e443..b06058d2f2f90 100644 --- a/tidb-cloud/premium/import-csv-files-premium.md +++ b/tidb-cloud/premium/import-csv-files-premium.md @@ -32,7 +32,7 @@ To ensure data consistency, {{{ .premium }}} allows importing CSV files into emp > > - To achieve better performance, it is recommended to limit the size of each compressed file to 100 MiB. > - The Snappy compressed file must be in the [official Snappy format](https://github.com/google/snappy). Other variants of Snappy compression are not supported. - > - For uncompressed files, if you cannot update the CSV filenames according to the preceding rules in some cases (for example, the CSV file links are also used by your other programs), you can keep the filenames unchanged and use the **Mapping Settings** in [Step 4](#step-4-import-csv-files) to import your source data to a single target table. + > - For uncompressed files, if you cannot update the CSV filenames according to the preceding rules in some cases (for example, the CSV file links are also used by your other programs), you can keep the filenames unchanged and configure manual source-to-target mappings in [Step 4](#step-4-import-csv-files). ## Step 2. Create the target table schemas @@ -205,10 +205,10 @@ If you get an importing error, do the following: ### Resolve warnings during data import -After clicking **Start Import**, if you see a warning message such as `can't find the corresponding source files`, resolve this by providing the correct source file, renaming the existing one according to [Naming Conventions for Data Import](/tidb-cloud/naming-conventions-for-data-import.md), or using **Advanced Settings** to make changes. +If the pre-check shows a warning such as `can't find the corresponding source files`, resolve it by providing the correct source file, renaming the existing one according to [Naming Conventions for Data Import](/tidb-cloud/naming-conventions-for-data-import.md), or returning to **Source Files Mapping** and configuring manual mappings. -After resolving these issues, you need to import the data again. +After resolving the issues, run the pre-check again. ### Zero rows in the imported tables -After the import progress shows **Completed**, check the imported tables. If the number of rows is zero, it means no data files matched the Bucket URI that you entered. In this case, resolve this issue by providing the correct source file, renaming the existing one according to [Naming Conventions for Data Import](/tidb-cloud/naming-conventions-for-data-import.md), or using **Advanced Settings** to make changes. After that, import those tables again. +After the import progress shows **Completed**, check the imported tables. If the number of rows is zero, verify that the **Source Files URI** and manual source patterns match the intended files. Correct the URI or mappings, and then import the tables again. From 096fcf369f8a6f5576ddb7d046dbc4733d0bc9bd Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Wed, 12 Aug 2026 15:23:17 -0400 Subject: [PATCH 3/8] docs: align Premium import source guidance --- .../premium/import-csv-files-premium.md | 20 ++++--------------- tidb-cloud/premium/import-from-s3-premium.md | 5 ++--- 2 files changed, 6 insertions(+), 19 deletions(-) diff --git a/tidb-cloud/premium/import-csv-files-premium.md b/tidb-cloud/premium/import-csv-files-premium.md index b06058d2f2f90..b5220b6f8e46d 100644 --- a/tidb-cloud/premium/import-csv-files-premium.md +++ b/tidb-cloud/premium/import-csv-files-premium.md @@ -105,9 +105,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 3. On the **Import Data from Cloud Storage** page, provide the following information: - **Storage Provider**: select **Amazon S3**. - - **Source Files URI**: - - When importing one file, enter the source file URI in the following format `s3://[bucket_name]/[data_source_folder]/[file_name].csv`. For example, `s3://sampledata/ingest/TableName.01.csv`. - - When importing multiple files, enter the source folder URI in the following format `s3://[bucket_name]/[data_source_folder]/`. For example, `s3://sampledata/ingest/`. + - **Source Files URI**: enter the URI of the top-level folder that contains the source files. For example, `s3://sampledata/ingest/`. - **Credential**: you can use either an AWS Role ARN or an AWS access key to access your bucket. For more information, see [Configure Amazon S3 access](/tidb-cloud/configure-external-storage-access.md#configure-amazon-s3-access). - **AWS Role ARN**: enter the AWS Role ARN value. If you need to create a new role, click **Click here to create a new one with AWS CloudFormation** and follow the guided steps to launch the provided template, acknowledge the IAM warning, create the stack, and copy the generated ARN back into {{{ .premium }}}. - **AWS Access Key**: enter the AWS access key ID and AWS secret access key. @@ -118,11 +116,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. - When a directory is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. - - > **Note:** - > - > When a single file is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is not displayed, and {{{ .premium }}} automatically populates the **Source** field with the file name. In this case, you only need to select the target database and table for data import. + The **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. @@ -157,9 +151,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 3. On the **Import Data from Cloud Storage** page, provide the following information: - **Storage Provider**: select **Alibaba Cloud OSS**. - - **Source Files URI**: - - When importing one file, enter the source file URI in the following format `oss://[bucket_name]/[data_source_folder]/[file_name].csv`. For example, `oss://sampledata/ingest/TableName.01.csv`. - - When importing multiple files, enter the source folder URI in the following format `oss://[bucket_name]/[data_source_folder]/`. For example, `oss://sampledata/ingest/`. + - **Source Files URI**: enter the URI of the top-level folder that contains the source files. For example, `oss://sampledata/ingest/`. - **Credential**: you can use an AccessKey pair to access your bucket. For more information, see [Configure Alibaba Cloud Object Storage Service (OSS) access](/tidb-cloud/configure-external-storage-access.md#configure-alibaba-cloud-object-storage-service-oss-access). - **Test Bucket Access**: click this button after the credentials are in place to confirm that {{{ .premium }}} can reach the bucket. - **Target Connection**: provide the TiDB username and password that will run the import. Optionally, click **Test Connection** to validate the credentials. @@ -168,11 +160,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. - When a directory is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. - - > **Note:** - > - > When a single file is specified in **Source Files URI**, the **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is not displayed, and {{{ .premium }}} automatically populates the **Source** field with the file name. In this case, you only need to select the target database and table for data import. + The **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. diff --git a/tidb-cloud/premium/import-from-s3-premium.md b/tidb-cloud/premium/import-from-s3-premium.md index 03319b7bad303..028dafbb54324 100644 --- a/tidb-cloud/premium/import-from-s3-premium.md +++ b/tidb-cloud/premium/import-from-s3-premium.md @@ -15,7 +15,6 @@ This document describes how to import CSV files from Amazon Simple Storage Servi ## Limitations - To ensure data consistency, {{{ .premium }}} allows importing CSV files into empty tables only. If the target table already contains data, import into a staging table and then copy the rows using the `INSERT ... SELECT` statement. -- During the public preview, the user interface currently supports Amazon S3 as the only storage provider. Support for additional providers will be added in future releases. - Each source pattern maps to one destination table. You can add multiple mappings to an import job. ## Step 1. Prepare the CSV files @@ -50,9 +49,9 @@ The wizard includes a helper link labeled **Click here to create a new one with 2. In the left navigation pane, click **Data** > **Import**, and choose **Import data from Cloud Storage**. 3. In the **Source Connection** dialog: - Set **Storage Provider** to **Amazon S3**. - - Enter the **Source Files URI** for a single file (`s3://bucket/path/file.csv`) or for a folder (`s3://bucket/path/`). + - Enter the **Source Files URI** for the top-level folder that contains the source files, for example, `s3://bucket/path/`. - Choose **AWS Role ARN** or **AWS Access Key** and provide the credentials. - - Click **Test Bucket Access** to validate connectivity. + - Click **Test Bucket Access** to validate connectivity. 4. Click **Next** and provide the TiDB SQL username and password for the import job. Optionally, test the connection. 5. Configure the source-to-target mapping: From 827aa738f85280c06ba3f5708df868569d0ce6b0 Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Wed, 12 Aug 2026 15:33:48 -0400 Subject: [PATCH 4/8] docs: match Premium mapping option label --- tidb-cloud/premium/import-csv-files-premium.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tidb-cloud/premium/import-csv-files-premium.md b/tidb-cloud/premium/import-csv-files-premium.md index b5220b6f8e46d..d4da68d1d1f1a 100644 --- a/tidb-cloud/premium/import-csv-files-premium.md +++ b/tidb-cloud/premium/import-csv-files-premium.md @@ -116,9 +116,9 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. - The **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. + The **Use [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. - - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. + - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: @@ -160,9 +160,9 @@ To import the CSV files to {{{ .premium }}}, take the following steps: 5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. - The **Use [File naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. + The **Use [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. - - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. + - To use automatic mapping, leave the option selected. {{{ .premium }}} applies the [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) to your source files and target tables. - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: From 4f7dbfedda8c78a59d1821bfaeb744092aaa0af0 Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Wed, 12 Aug 2026 15:40:20 -0400 Subject: [PATCH 5/8] docs: scope Premium mapping recovery guidance --- tidb-cloud/naming-conventions-for-data-import.md | 2 +- tidb-cloud/premium/import-csv-files-premium.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tidb-cloud/naming-conventions-for-data-import.md b/tidb-cloud/naming-conventions-for-data-import.md index 609a0c81bcaee..8bc8e55388bae 100644 --- a/tidb-cloud/naming-conventions-for-data-import.md +++ b/tidb-cloud/naming-conventions-for-data-import.md @@ -132,7 +132,7 @@ Manual file-pattern mapping does not support SQL data files. -In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern relative to the source URI and supports the `*` and `?` wildcards. Add one mapping for each target table. +In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern relative to the source URI and supports the `*` and `?` wildcards. diff --git a/tidb-cloud/premium/import-csv-files-premium.md b/tidb-cloud/premium/import-csv-files-premium.md index d4da68d1d1f1a..c176802ab4c1e 100644 --- a/tidb-cloud/premium/import-csv-files-premium.md +++ b/tidb-cloud/premium/import-csv-files-premium.md @@ -199,4 +199,4 @@ After resolving the issues, run the pre-check again. ### Zero rows in the imported tables -After the import progress shows **Completed**, check the imported tables. If the number of rows is zero, verify that the **Source Files URI** and manual source patterns match the intended files. Correct the URI or mappings, and then import the tables again. +After the import progress shows **Completed**, check the imported tables. If the number of rows is zero, verify that the **Source Files URI** is correct and that the source files either follow the [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping or match the manual source patterns. Correct the URI, rename the files, or configure manual mappings, and then import the tables again. From 840da9002fc52cdb1cc6620fba7f1aac70a3d577 Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Wed, 12 Aug 2026 15:46:48 -0400 Subject: [PATCH 6/8] docs: align Premium mapping terminology --- tidb-cloud/premium/import-csv-files-premium.md | 4 ++-- tidb-cloud/premium/import-from-s3-premium.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tidb-cloud/premium/import-csv-files-premium.md b/tidb-cloud/premium/import-csv-files-premium.md index c176802ab4c1e..08607d91e8d94 100644 --- a/tidb-cloud/premium/import-csv-files-premium.md +++ b/tidb-cloud/premium/import-csv-files-premium.md @@ -123,7 +123,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `?` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`. - - **Target Database** and **Target Table**: enter the destination database and table for the matched files. + - **Target Database** and **Target Table**: enter the target database and table for the matched files. - To add another mapping, click **+**. 6. Click **Next** to run the pre-check. Review the scan results and verify the source files and target tables. @@ -167,7 +167,7 @@ To import the CSV files to {{{ .premium }}}, take the following steps: - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `?` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`. - - **Target Database** and **Target Table**: enter the destination database and table for the matched files. + - **Target Database** and **Target Table**: enter the target database and table for the matched files. - To add another mapping, click **+**. 6. Click **Next** to run the pre-check. Review the scan results and verify the source files and target tables. diff --git a/tidb-cloud/premium/import-from-s3-premium.md b/tidb-cloud/premium/import-from-s3-premium.md index 028dafbb54324..678415345336a 100644 --- a/tidb-cloud/premium/import-from-s3-premium.md +++ b/tidb-cloud/premium/import-from-s3-premium.md @@ -15,7 +15,7 @@ This document describes how to import CSV files from Amazon Simple Storage Servi ## Limitations - To ensure data consistency, {{{ .premium }}} allows importing CSV files into empty tables only. If the target table already contains data, import into a staging table and then copy the rows using the `INSERT ... SELECT` statement. -- Each source pattern maps to one destination table. You can add multiple mappings to an import job. +- Each source pattern maps to one target table. You can add multiple mappings to an import job. ## Step 1. Prepare the CSV files From 64da2d4a32fb817603c71ce8df58d02d2dfea22e Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Thu, 20 Aug 2026 18:57:06 -0400 Subject: [PATCH 7/8] docs: correct import wildcard guidance --- sql-statements/sql-statement-import-into.md | 7 +++- tidb-cloud/import-csv-files-serverless.md | 20 ++++++----- tidb-cloud/import-csv-files.md | 9 +++-- tidb-cloud/import-parquet-files-serverless.md | 20 ++++++----- tidb-cloud/import-parquet-files.md | 9 +++-- tidb-cloud/limited-sql-features-tidb-x.md | 2 +- tidb-cloud/migrate-sql-shards.md | 12 +++---- .../naming-conventions-for-data-import.md | 6 +++- .../premium/import-csv-files-premium.md | 30 ++++++++++++----- tidb-cloud/premium/import-from-s3-premium.md | 33 ++++++++++++------- 10 files changed, 96 insertions(+), 52 deletions(-) diff --git a/sql-statements/sql-statement-import-into.md b/sql-statements/sql-statement-import-into.md index 86040ff8d6b11..9f585caa9c426 100644 --- a/sql-statements/sql-statement-import-into.md +++ b/sql-statements/sql-statement-import-into.md @@ -121,7 +121,11 @@ It specifies the storage location of the data file, which can be an Amazon S3 or > > If [SEM](/system-variables.md#tidb_enable_enhanced_security) is enabled in the target cluster, the `fileLocation` cannot be specified as a local file path. -In the `fileLocation` parameter, you can specify a single file, or use the `*` and `[]` wildcards to match multiple files for import. Note that the wildcard can only be used in the file name, because it does not match directories or recursively match files in subdirectories. Taking files stored on Amazon S3 as examples, you can configure the parameter as follows: +In the `fileLocation` parameter, you can specify a single file or use the `*` and `[]` wildcards to match multiple files for import. TiDB treats `?` as a literal character, not a wildcard. The `[]` wildcard is supported starting from TiDB v8.0.0. + +For files in Amazon S3 or GCS, wildcards can appear in multiple fixed-depth path components. A wildcard matches characters only within one `/`-separated component and does not cross `/`. Recursive matching across an arbitrary number of directory levels, such as with `**`, is not supported. For files in TiDB local storage, the directory path must be exact, and wildcards can appear only in the file name. + +Taking files stored on Amazon S3 as examples, you can configure the parameter as follows: - Import a single file: `s3:///path/to/data/foo.csv` - Import all files in a specified path: `s3:///path/to/data/*` @@ -129,6 +133,7 @@ In the `fileLocation` parameter, you can specify a single file, or use the `*` a - Import all files with the `foo` prefix in a specified path: `s3:///path/to/data/foo*` - Import all files with the `foo` prefix and the `.csv` suffix in a specified path: `s3:///path/to/data/foo*.csv` - Import `1.csv` and `2.csv` in a specified path: `s3:///path/to/data/[12].csv` +- Import all CSV files from fixed-depth subdirectories whose names start with `data-`: `s3:///path/to/data-*/*.csv` ### Format diff --git a/tidb-cloud/import-csv-files-serverless.md b/tidb-cloud/import-csv-files-serverless.md index d2fb046d6fd73..8302bbf0c28bc 100644 --- a/tidb-cloud/import-csv-files-serverless.md +++ b/tidb-cloud/import-csv-files-serverless.md @@ -129,9 +129,10 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.csv`: matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. - **Target Database** and **Target Table**: select the target database and table to import the data to. @@ -180,9 +181,10 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.csv`: matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. - **Target Database** and **Target Table**: select the target database and table to import the data to. @@ -231,9 +233,10 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.csv`: matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. - **Target Database** and **Target Table**: select the target database and table to import the data to. @@ -282,9 +285,10 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.csv`: matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. - **Target Database** and **Target Table**: select the target database and table to import the data to. diff --git a/tidb-cloud/import-csv-files.md b/tidb-cloud/import-csv-files.md index 6aa4601c85f06..860b63f8ebee8 100644 --- a/tidb-cloud/import-csv-files.md +++ b/tidb-cloud/import-csv-files.md @@ -135,10 +135,11 @@ To import the CSV files to TiDB Cloud, take the following steps: - To manually configure the mapping rules to associate your source CSV files with the target database and table, deselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example, `TableName.01.csv`. You can also use wildcards to match multiple files. TiDB Cloud only supports the `*` and `?` wildcards. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example, `TableName.01.csv`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files. - `my-data?.csv`: matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. + - `my-data[1-4].csv`: matches CSV files named `my-data1.csv` through `my-data4.csv`. - **Target Database** and **Target Table**: enter the target database and table to import the data to. @@ -188,10 +189,11 @@ To import the CSV files to TiDB Cloud, take the following steps: - To manually configure the mapping rules to associate your source CSV files with the target database and table, deselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example, `TableName.01.csv`. You can also use wildcards to match multiple files. TiDB Cloud only supports the `*` and `?` wildcards. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example, `TableName.01.csv`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files. - `my-data?.csv`: matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. + - `my-data[1-4].csv`: matches CSV files named `my-data1.csv` through `my-data4.csv`. - **Target Database** and **Target Table**: enter the target database and table to import the data to. @@ -261,10 +263,11 @@ To import the CSV files to TiDB Cloud, take the following steps: - To manually configure the mapping rules to associate your source CSV files with the target database and table, deselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example, `TableName.01.csv`. You can also use wildcards to match multiple files. TiDB Cloud only supports the `*` and `?` wildcards. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example, `TableName.01.csv`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files. - `my-data?.csv`: matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. + - `my-data[1-4].csv`: matches CSV files named `my-data1.csv` through `my-data4.csv`. - **Target Database** and **Target Table**: enter the target database and table to import the data to. diff --git a/tidb-cloud/import-parquet-files-serverless.md b/tidb-cloud/import-parquet-files-serverless.md index 7cb3936821dd9..87a836e1c439a 100644 --- a/tidb-cloud/import-parquet-files-serverless.md +++ b/tidb-cloud/import-parquet-files-serverless.md @@ -134,9 +134,10 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.parquet`: matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. - **Target Database** and **Target Table**: select the target database and table to import the data to. @@ -185,9 +186,10 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.parquet`: matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. - **Target Database** and **Target Table**: select the target database and table to import the data to. @@ -236,9 +238,10 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.parquet`: matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. - **Target Database** and **Target Table**: select the target database and table to import the data to. @@ -287,9 +290,10 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. Only `*` and `?` wildcards are supported. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. - - `my-data?.parquet`: matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. - **Target Database** and **Target Table**: select the target database and table to import the data to. diff --git a/tidb-cloud/import-parquet-files.md b/tidb-cloud/import-parquet-files.md index 915c568589b7b..514325f6827cd 100644 --- a/tidb-cloud/import-parquet-files.md +++ b/tidb-cloud/import-parquet-files.md @@ -141,10 +141,11 @@ To import the Parquet files to TiDB Cloud, take the following steps: - To manually configure the mapping rules to associate your source Parquet files with the target database and table, deselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example, `TableName.01.parquet`. You can also use wildcards to match multiple files. TiDB Cloud only supports the `*` and `?` wildcards. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example, `TableName.01.parquet`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files. - `my-data?.parquet`: matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data10.parquet` and `my-data100.parquet`. + - `my-data[1-4].parquet`: matches Parquet files named `my-data1.parquet` through `my-data4.parquet`. - **Target Database** and **Target Table**: enter the target database and table to import the data to. @@ -192,10 +193,11 @@ To import the Parquet files to TiDB Cloud, take the following steps: - To manually configure the mapping rules to associate your source Parquet files with the target database and table, deselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example, `TableName.01.parquet`. You can also use wildcards to match multiple files. TiDB Cloud only supports the `*` and `?` wildcards. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example, `TableName.01.parquet`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files. - `my-data?.parquet`: matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data10.parquet` and `my-data100.parquet`. + - `my-data[1-4].parquet`: matches Parquet files named `my-data1.parquet` through `my-data4.parquet`. - **Target Database** and **Target Table**: enter the target database and table to import the data to. @@ -263,10 +265,11 @@ To import the Parquet files to TiDB Cloud, take the following steps: - To manually configure the mapping rules to associate your source Parquet files with the target database and table, deselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example, `TableName.01.parquet`. You can also use wildcards to match multiple files. TiDB Cloud only supports the `*` and `?` wildcards. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example, `TableName.01.parquet`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files. - `my-data?.parquet`: matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data10.parquet` and `my-data100.parquet`. + - `my-data[1-4].parquet`: matches Parquet files named `my-data1.parquet` through `my-data4.parquet`. - **Target Database** and **Target Table**: enter the target database and table to import the data to. diff --git a/tidb-cloud/limited-sql-features-tidb-x.md b/tidb-cloud/limited-sql-features-tidb-x.md index 834f4f1e361dd..c3d3dfb9c2956 100644 --- a/tidb-cloud/limited-sql-features-tidb-x.md +++ b/tidb-cloud/limited-sql-features-tidb-x.md @@ -76,7 +76,7 @@ TiDB Cloud works with almost all workloads that TiDB supports, but there are som | `CHANGE DRAINER` | Not supported [^7] | Not supported [^7] | | `CHANGE PUMP` | Not supported [^7] | Not supported [^7] | | `FLASHBACK CLUSTER` | Not supported [^3] | Not supported [^3] | -| `IMPORT INTO` | Supported, but blocks IMPORT INTO statements that use a local file path or an S3 EXTERNAL_ID. | Not supported | +| `IMPORT INTO` | Supported, but blocks `IMPORT INTO` statements that use a local file path or an S3 `EXTERNAL_ID`. | Supported on {{{ .starter }}} and Essential with the same restrictions as Premium. | | `LOAD DATA INFILE` | Blocks `LOAD DATA LOCAL INFILE` from a local file path. | Only supports `LOAD DATA LOCAL INFILE` | | `LOAD STATS` | Not supported | Not supported | | `SELECT ... INTO OUTFILE` | Not supported [^4] | Not supported [^4] | diff --git a/tidb-cloud/migrate-sql-shards.md b/tidb-cloud/migrate-sql-shards.md index 51222160cbfc2..a8994f2658df8 100644 --- a/tidb-cloud/migrate-sql-shards.md +++ b/tidb-cloud/migrate-sql-shards.md @@ -205,19 +205,17 @@ After configuring the Amazon S3 access, you can perform the data import task in When importing multiple files, you can use **Advanced Settings** > **Mapping Settings** to define a custom mapping rule for each target table and its corresponding CSV file. After that, the data source files will be re-scanned using the provided custom mapping rule. - When you enter the source file URI and name in **Source File URIs and Names**, make sure it is in the following format `s3://[bucket_name]/[data_source_folder]/[file_name].csv`. For example, `s3://sampledata/ingest/TableName.01.csv`. + The source-pattern field and supported wildcards depend on the service plan: - You can also use wildcards to match the source files. For example: + - For {{{ .starter }}} and {{{ .essential }}} instances that display **Destination Mapping**, enter a file name pattern relative to the top-level source URI. Use `*`, `?`, or `[]`. For example, `my-data?.csv` matches files such as `my-data1.csv` and `my-data2.csv`, `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`, and `my-data*.csv` matches all CSV files whose names start with `my-data`. - - `s3://[bucket_name]/[data_source_folder]/my-data?.csv`: all CSV files starting with `my-data` followed by one character (such as `my-data1.csv` and `my-data2.csv`) in that folder will be imported into the same target table. + - For {{{ .essential }}} and {{{ .premium }}} instances that display **Mapping and Job Configuration**, deselect **Use TiDB file naming conventions for automatic mapping**. In **Source**, enter a file name pattern relative to the **Source Files URI**. Use `[]` to match a character range and `*` to match multiple characters. For example, `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`, and `my-data*.csv` matches all CSV files whose names start with `my-data`. - - `s3://[bucket_name]/[data_source_folder]/my-data*.csv`: all CSV files in the folder starting with `my-data` will be imported into the same target table. - - Note that only `?` and `*` are supported. + - For {{{ .dedicated }}}, enter the full source file URI in **Source File URIs and Names**. Use `?` to match one character, `*` to match multiple characters, and `[]` to match a character range. For example, `s3://[bucket_name]/[data_source_folder]/my-data?.csv` matches files such as `my-data1.csv` and `my-data2.csv`, `s3://[bucket_name]/[data_source_folder]/my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`, and `s3://[bucket_name]/[data_source_folder]/my-data*.csv` matches all CSV files in the folder whose names start with `my-data`. > **Note:** > - > The URI must contain the data source folder. + > In the **Destination Mapping** and **Source Files Mapping** workflows, configure the top-level folder in the source URI field, and enter only the relative file name pattern in each mapping. For {{{ .dedicated }}}, the URI in each mapping must contain the data source folder. 6. Edit the CSV configuration if needed. diff --git a/tidb-cloud/naming-conventions-for-data-import.md b/tidb-cloud/naming-conventions-for-data-import.md index 8bc8e55388bae..6819eeeac0641 100644 --- a/tidb-cloud/naming-conventions-for-data-import.md +++ b/tidb-cloud/naming-conventions-for-data-import.md @@ -132,7 +132,11 @@ Manual file-pattern mapping does not support SQL data files. -In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern relative to the source URI and supports the `*` and `?` wildcards. +In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern relative to the source URI. The supported wildcards depend on the service plan: + +- For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For example, `my-data?.csv` matches file names such as `my-data1.csv` and `my-data2.csv`, and `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. +- For {{{ .dedicated }}}, use `*`, `?`, and `[]`. For example, `my-data?.csv` matches file names such as `my-data1.csv` and `my-data2.csv`, and `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. +- For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. For example, `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. diff --git a/tidb-cloud/premium/import-csv-files-premium.md b/tidb-cloud/premium/import-csv-files-premium.md index 08607d91e8d94..9c90ddbcd768c 100644 --- a/tidb-cloud/premium/import-csv-files-premium.md +++ b/tidb-cloud/premium/import-csv-files-premium.md @@ -106,15 +106,15 @@ To import the CSV files to {{{ .premium }}}, take the following steps: - **Storage Provider**: select **Amazon S3**. - **Source Files URI**: enter the URI of the top-level folder that contains the source files. For example, `s3://sampledata/ingest/`. - - **Credential**: you can use either an AWS Role ARN or an AWS access key to access your bucket. For more information, see [Configure Amazon S3 access](/tidb-cloud/configure-external-storage-access.md#configure-amazon-s3-access). - - **AWS Role ARN**: enter the AWS Role ARN value. If you need to create a new role, click **Click here to create a new one with AWS CloudFormation** and follow the guided steps to launch the provided template, acknowledge the IAM warning, create the stack, and copy the generated ARN back into {{{ .premium }}}. - - **AWS Access Key**: enter the AWS access key ID and AWS secret access key. + - **Credentials**: you can use either an AWS Role ARN or an AWS access key to access your bucket. For more information, see [Configure Amazon S3 access](/tidb-cloud/configure-external-storage-access.md#configure-amazon-s3-access). + - **AWS Role ARN**: enter the AWS Role ARN value. If you need to create a new role, click **Click here to create new one with AWS CloudFormation** and follow the guided steps to launch the provided template, acknowledge the IAM warning, create the stack, and copy the generated ARN back into {{{ .premium }}}. + - **AWS Access Key**: enter the **Access Key ID** and **Secret Access Key**. - **Test Bucket Access**: click this button after the credentials are in place to confirm that {{{ .premium }}} can reach the bucket. - **Target Connection**: provide the TiDB username and password that will run the import. Optionally, click **Test Connection** to validate the credentials. 4. Click **Next**. -5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. +5. In the **Mapping and Job Configuration** step, use **Source Files Mapping** to specify how source files are mapped to target tables. The **Use [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. @@ -122,12 +122,19 @@ To import the CSV files to {{{ .premium }}}, take the following steps: - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: - - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `?` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`. + - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `[]` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`, and `my-data-[1-4].csv` matches `my-data-1.csv` through `my-data-4.csv`. - **Target Database** and **Target Table**: enter the target database and table for the matched files. - To add another mapping, click **+**. + In the **Job Configuration** section, review the following settings: + + - **Job Name**: keep the generated name or enter a name for the import job. + - **Advanced Options**: expand this section to view **Ignore compatibility checks (advanced)**. Leave this option disabled unless you intentionally need to bypass the pre-import compatibility checks. + 6. Click **Next** to run the pre-check. Review the scan results and verify the source files and target tables. + By default, **Skip first** is `0`. If each source CSV file contains a header row, click **edit CSV configuration here** and set **Skip first** to `1`. This setting applies to every matched CSV file. + 7. Click **Start Import**. 8. When the import progress shows **Completed**, check the imported tables. @@ -152,13 +159,13 @@ To import the CSV files to {{{ .premium }}}, take the following steps: - **Storage Provider**: select **Alibaba Cloud OSS**. - **Source Files URI**: enter the URI of the top-level folder that contains the source files. For example, `oss://sampledata/ingest/`. - - **Credential**: you can use an AccessKey pair to access your bucket. For more information, see [Configure Alibaba Cloud Object Storage Service (OSS) access](/tidb-cloud/configure-external-storage-access.md#configure-alibaba-cloud-object-storage-service-oss-access). + - **AccessKey ID** and **AccessKey Secret**: enter an AccessKey pair to access your bucket. For more information, see [Configure Alibaba Cloud Object Storage Service (OSS) access](/tidb-cloud/configure-external-storage-access.md#configure-alibaba-cloud-object-storage-service-oss-access). - **Test Bucket Access**: click this button after the credentials are in place to confirm that {{{ .premium }}} can reach the bucket. - **Target Connection**: provide the TiDB username and password that will run the import. Optionally, click **Test Connection** to validate the credentials. 4. Click **Next**. -5. In the **Source Files Mapping** section, specify how source files are mapped to target tables. +5. In the **Mapping and Job Configuration** step, use **Source Files Mapping** to specify how source files are mapped to target tables. The **Use [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md) for automatic mapping** option is selected by default. @@ -166,12 +173,19 @@ To import the CSV files to {{{ .premium }}}, take the following steps: - To configure mappings manually, deselect the automatic mapping option, and then configure a mapping for each target table: - - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `?` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`. + - **Source**: enter a source file pattern relative to the **Source Files URI**. The pattern supports `*` and `[]` wildcards. For example, `my-data*.csv` matches all CSV files whose names start with `my-data`, and `my-data-[1-4].csv` matches `my-data-1.csv` through `my-data-4.csv`. - **Target Database** and **Target Table**: enter the target database and table for the matched files. - To add another mapping, click **+**. + In the **Job Configuration** section, review the following settings: + + - **Job Name**: keep the generated name or enter a name for the import job. + - **Advanced Options**: expand this section to view **Ignore compatibility checks (advanced)**. Leave this option disabled unless you intentionally need to bypass the pre-import compatibility checks. + 6. Click **Next** to run the pre-check. Review the scan results and verify the source files and target tables. + By default, **Skip first** is `0`. If each source CSV file contains a header row, click **edit CSV configuration here** and set **Skip first** to `1`. This setting applies to every matched CSV file. + 7. Click **Start Import**. 8. When the import progress shows **Completed**, check the imported tables. diff --git a/tidb-cloud/premium/import-from-s3-premium.md b/tidb-cloud/premium/import-from-s3-premium.md index 678415345336a..e2d8d613e2ba2 100644 --- a/tidb-cloud/premium/import-from-s3-premium.md +++ b/tidb-cloud/premium/import-from-s3-premium.md @@ -20,15 +20,14 @@ This document describes how to import CSV files from Amazon Simple Storage Servi ## Step 1. Prepare the CSV files 1. If a CSV file is larger than 256 MiB, consider splitting it into smaller files around 256 MiB so {{{ .premium }}} can process them in parallel. -2. Name your CSV files according to the Dumpling naming conventions: +2. If you plan to use automatic mapping, name your CSV files according to the [TiDB file naming conventions](/tidb-cloud/naming-conventions-for-data-import.md): - Full-table files: use the `${db_name}.${table_name}.csv` format. - Sharded files: append numeric suffixes, such as `${db_name}.${table_name}.000001.csv`. - Compressed files: use the `${db_name}.${table_name}.${suffix}.csv.${compress}` format. -3. Optional schema files (`${db_name}-schema-create.sql`, `${db_name}.${table_name}-schema.sql`) help {{{ .premium }}} create databases and tables automatically. - + If you plan to configure mappings manually, you can keep the existing file names and specify source patterns in [Step 4](#step-4-import-csv-files-from-amazon-s3). + +3. Optional schema files (`${db_name}-schema-create.sql`, `${db_name}.${table_name}-schema.sql`) help {{{ .premium }}} create databases and tables automatically. ## Step 2. Create target schemas (optional) @@ -41,29 +40,39 @@ To allow {{{ .premium }}} to read your bucket, use either of the following metho - Provide an AWS Role ARN that trusts TiDB Cloud and grants the `s3:GetObject` and `s3:ListBucket` permissions on the relevant paths. - Provide an AWS access key (access key ID and secret access key) with equivalent permissions. -The wizard includes a helper link labeled **Click here to create a new one with AWS CloudFormation**. Follow this link if you need {{{ .premium }}} to pre-fill a CloudFormation stack that creates the role for you. +The wizard includes a helper link labeled **Click here to create new one with AWS CloudFormation**. Follow this link if you need {{{ .premium }}} to pre-fill a CloudFormation stack that creates the role for you. ## Step 4. Import CSV files from Amazon S3 1. In the [TiDB Cloud console](https://tidbcloud.com/tidbs), navigate to the [**My TiDB**](https://tidbcloud.com/tidbs) page, and then click the name of your {{{ .premium }}} instance. 2. In the left navigation pane, click **Data** > **Import**, and choose **Import data from Cloud Storage**. -3. In the **Source Connection** dialog: +3. In the **Source and Target Connection** step: - Set **Storage Provider** to **Amazon S3**. - Enter the **Source Files URI** for the top-level folder that contains the source files, for example, `s3://bucket/path/`. - - Choose **AWS Role ARN** or **AWS Access Key** and provide the credentials. + - Under **Credentials**, choose **AWS Role ARN** and enter the role ARN, or choose **AWS Access Key** and enter the **Access Key ID** and **Secret Access Key**. - Click **Test Bucket Access** to validate connectivity. + - Under **Target Connection**, provide the TiDB SQL username and password for the import job. Optionally, click **Test Connection**. -4. Click **Next** and provide the TiDB SQL username and password for the import job. Optionally, test the connection. -5. Configure the source-to-target mapping: +4. Click **Next**. +5. In the **Mapping and Job Configuration** step, use **Source Files Mapping** to configure the source-to-target mapping: - To use automatic mapping, leave **Use TiDB file naming conventions for automatic mapping** selected. - - To configure mappings manually, deselect the automatic mapping option. For each target table, enter a source file pattern relative to the **Source Files URI**, and then enter the target database and table. The source pattern supports `*` and `?` wildcards. To add another mapping, click **+**. + - To configure mappings manually, deselect the automatic mapping option. For each target table, enter a source file pattern relative to the **Source Files URI**, and then enter the target database and table. The source pattern supports `*` and `[]` wildcards. For example, `my-data-[1-4].csv` matches `my-data-1.csv` through `my-data-4.csv`. To add another mapping, click **+**. + + In the **Job Configuration** section: + + - Keep the generated **Job Name** or enter a name for the import job. + - Expand **Advanced Options** to view **Ignore compatibility checks (advanced)**. Leave this option disabled unless you intentionally need to bypass the pre-import compatibility checks. + 6. Click **Next** to run the pre-check. Resolve any warnings about missing files or incompatible schemas. + + By default, **Skip first** is `0`. If each source CSV file contains a header row, click **edit CSV configuration here** and set **Skip first** to `1`. This setting applies to every matched CSV file. + 7. Click **Start Import** to launch the job group. 8. Monitor the job statuses until they show **Completed**, then verify the imported data in TiDB Cloud. ## Troubleshooting -- If the pre-check reports zero files, verify the S3 path and IAM permissions. +- If the pre-check reports zero files, verify the **Source Files URI**, IAM permissions, and manual source patterns. - If jobs remain in **Preparing**, ensure that the destination tables are empty and the required schema files exist. - Use the **Cancel** action to stop a job group if you need to adjust mappings or credentials. From 1c1bc63bf86737fcc0c820086b1edb5637188d1c Mon Sep 17 00:00:00 2001 From: Airton Lastori <6343615+alastori@users.noreply.github.com> Date: Thu, 20 Aug 2026 19:20:43 -0400 Subject: [PATCH 8/8] docs: scope import wildcard guidance by workflow --- tidb-cloud/import-csv-files-serverless.md | 16 ++++++++-------- tidb-cloud/import-parquet-files-serverless.md | 16 ++++++++-------- tidb-cloud/limited-sql-features-tidb-x.md | 2 +- tidb-cloud/migrate-sql-shards.md | 12 +++++++----- tidb-cloud/naming-conventions-for-data-import.md | 7 +++---- 5 files changed, 27 insertions(+), 26 deletions(-) diff --git a/tidb-cloud/import-csv-files-serverless.md b/tidb-cloud/import-csv-files-serverless.md index 8302bbf0c28bc..e43c9423ee8df 100644 --- a/tidb-cloud/import-csv-files-serverless.md +++ b/tidb-cloud/import-csv-files-serverless.md @@ -129,9 +129,9 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. @@ -181,9 +181,9 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. @@ -233,9 +233,9 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. @@ -285,9 +285,9 @@ To import the CSV files to {{{ .starter }}} or {{{ .essential }}}, take the foll - To manually configure the mapping rules to associate your source CSV files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].csv` format. For example: `TableName.01.csv`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. + - `my-data?.csv` matches all CSV files that start with `my-data` followed by a single character, such as `my-data1.csv` and `my-data2.csv`. - `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. - `my-data*.csv`: matches all CSV files that start with `my-data`, such as `my-data-2023.csv` and `my-data-final.csv`. diff --git a/tidb-cloud/import-parquet-files-serverless.md b/tidb-cloud/import-parquet-files-serverless.md index 87a836e1c439a..210c56e4c5564 100644 --- a/tidb-cloud/import-parquet-files-serverless.md +++ b/tidb-cloud/import-parquet-files-serverless.md @@ -134,9 +134,9 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. @@ -186,9 +186,9 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. @@ -238,9 +238,9 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. @@ -290,9 +290,9 @@ To import the Parquet files to {{{ .starter }}} or {{{ .essential }}}, take the - To manually configure the mapping rules to associate your source Parquet files with the target database and table, unselect this option, and then fill in the following fields: - - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use wildcards to match multiple files. For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. + - **Source**: enter the file name pattern in the `[file_name].parquet` format. For example: `TableName.01.parquet`. You can also use the `*`, `?`, and `[]` wildcards to match multiple files in **Destination Mapping**. - - For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. + - `my-data?.parquet` matches all Parquet files that start with `my-data` followed by a single character, such as `my-data1.parquet` and `my-data2.parquet`. - `my-data[1-4].parquet` matches `my-data1.parquet` through `my-data4.parquet`. - `my-data*.parquet`: matches all Parquet files that start with `my-data`, such as `my-data-2023.parquet` and `my-data-final.parquet`. diff --git a/tidb-cloud/limited-sql-features-tidb-x.md b/tidb-cloud/limited-sql-features-tidb-x.md index c3d3dfb9c2956..8c3e85f3af357 100644 --- a/tidb-cloud/limited-sql-features-tidb-x.md +++ b/tidb-cloud/limited-sql-features-tidb-x.md @@ -76,7 +76,7 @@ TiDB Cloud works with almost all workloads that TiDB supports, but there are som | `CHANGE DRAINER` | Not supported [^7] | Not supported [^7] | | `CHANGE PUMP` | Not supported [^7] | Not supported [^7] | | `FLASHBACK CLUSTER` | Not supported [^3] | Not supported [^3] | -| `IMPORT INTO` | Supported, but blocks `IMPORT INTO` statements that use a local file path or an S3 `EXTERNAL_ID`. | Supported on {{{ .starter }}} and Essential with the same restrictions as Premium. | +| `IMPORT INTO` | Supported, but blocks `IMPORT INTO` statements that use a local file path or an S3 `EXTERNAL_ID`. | Supported, but blocks `IMPORT INTO` statements that use a local file path or an S3 `EXTERNAL_ID`. | | `LOAD DATA INFILE` | Blocks `LOAD DATA LOCAL INFILE` from a local file path. | Only supports `LOAD DATA LOCAL INFILE` | | `LOAD STATS` | Not supported | Not supported | | `SELECT ... INTO OUTFILE` | Not supported [^4] | Not supported [^4] | diff --git a/tidb-cloud/migrate-sql-shards.md b/tidb-cloud/migrate-sql-shards.md index a8994f2658df8..51222160cbfc2 100644 --- a/tidb-cloud/migrate-sql-shards.md +++ b/tidb-cloud/migrate-sql-shards.md @@ -205,17 +205,19 @@ After configuring the Amazon S3 access, you can perform the data import task in When importing multiple files, you can use **Advanced Settings** > **Mapping Settings** to define a custom mapping rule for each target table and its corresponding CSV file. After that, the data source files will be re-scanned using the provided custom mapping rule. - The source-pattern field and supported wildcards depend on the service plan: + When you enter the source file URI and name in **Source File URIs and Names**, make sure it is in the following format `s3://[bucket_name]/[data_source_folder]/[file_name].csv`. For example, `s3://sampledata/ingest/TableName.01.csv`. - - For {{{ .starter }}} and {{{ .essential }}} instances that display **Destination Mapping**, enter a file name pattern relative to the top-level source URI. Use `*`, `?`, or `[]`. For example, `my-data?.csv` matches files such as `my-data1.csv` and `my-data2.csv`, `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`, and `my-data*.csv` matches all CSV files whose names start with `my-data`. + You can also use wildcards to match the source files. For example: - - For {{{ .essential }}} and {{{ .premium }}} instances that display **Mapping and Job Configuration**, deselect **Use TiDB file naming conventions for automatic mapping**. In **Source**, enter a file name pattern relative to the **Source Files URI**. Use `[]` to match a character range and `*` to match multiple characters. For example, `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`, and `my-data*.csv` matches all CSV files whose names start with `my-data`. + - `s3://[bucket_name]/[data_source_folder]/my-data?.csv`: all CSV files starting with `my-data` followed by one character (such as `my-data1.csv` and `my-data2.csv`) in that folder will be imported into the same target table. - - For {{{ .dedicated }}}, enter the full source file URI in **Source File URIs and Names**. Use `?` to match one character, `*` to match multiple characters, and `[]` to match a character range. For example, `s3://[bucket_name]/[data_source_folder]/my-data?.csv` matches files such as `my-data1.csv` and `my-data2.csv`, `s3://[bucket_name]/[data_source_folder]/my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`, and `s3://[bucket_name]/[data_source_folder]/my-data*.csv` matches all CSV files in the folder whose names start with `my-data`. + - `s3://[bucket_name]/[data_source_folder]/my-data*.csv`: all CSV files in the folder starting with `my-data` will be imported into the same target table. + + Note that only `?` and `*` are supported. > **Note:** > - > In the **Destination Mapping** and **Source Files Mapping** workflows, configure the top-level folder in the source URI field, and enter only the relative file name pattern in each mapping. For {{{ .dedicated }}}, the URI in each mapping must contain the data source folder. + > The URI must contain the data source folder. 6. Edit the CSV configuration if needed. diff --git a/tidb-cloud/naming-conventions-for-data-import.md b/tidb-cloud/naming-conventions-for-data-import.md index 6819eeeac0641..8ea89cd2250ef 100644 --- a/tidb-cloud/naming-conventions-for-data-import.md +++ b/tidb-cloud/naming-conventions-for-data-import.md @@ -132,11 +132,10 @@ Manual file-pattern mapping does not support SQL data files. -In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the **Source**, **Target Database**, and **Target Table** fields. The **Source** field accepts a file name pattern relative to the source URI. The supported wildcards depend on the service plan: +In the mapping step of the import wizard, deselect **Use TiDB file naming conventions for automatic mapping**, and then fill in the source pattern, target database, and target table fields. In **Destination Mapping** and **Source Files Mapping**, the **Source** field accepts a file name pattern relative to the source URI. In **Source File URIs and Names**, enter the full source file URI. The supported wildcards depend on the mapping workflow: -- For {{{ .starter }}} and {{{ .essential }}} instances that use **Destination Mapping**, use `*`, `?`, and `[]`. For example, `my-data?.csv` matches file names such as `my-data1.csv` and `my-data2.csv`, and `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. -- For {{{ .dedicated }}}, use `*`, `?`, and `[]`. For example, `my-data?.csv` matches file names such as `my-data1.csv` and `my-data2.csv`, and `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. -- For {{{ .essential }}} and {{{ .premium }}} instances that use **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. For example, `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. +- In **Destination Mapping** or **Source File URIs and Names**, use `*`, `?`, and `[]`. For example, `my-data?.csv` matches file names such as `my-data1.csv` and `my-data2.csv`, and `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`. +- In **Source Files Mapping** under **Mapping and Job Configuration**, use `*` and `[]`. For example, `my-data[1-4].csv` matches `my-data1.csv` through `my-data4.csv`.