Skip to content

[fix](fe) Detect Doris JDBC targets with stable server identity - #68321

Open
suxiaogang223 wants to merge 3 commits into
apache:masterfrom
suxiaogang223:codex/jdbc-doris-session-identity
Open

suxiaogang223 wants to merge 3 commits into
apache:masterfrom
suxiaogang223:codex/jdbc-doris-session-identity

Conversation

@suxiaogang223

Copy link
Copy Markdown
Member

What problem does this PR solve?

JDBC Catalog currently decides whether a MySQL-compatible endpoint is Doris by parsing version_comment. That value is display text and may change with product branding or packaging. When the expected marker is absent, the endpoint is treated as MySQL and Doris-specific types can be mapped incorrectly.

This change exposes a stable, read-only doris_server_identity session variable with the value apache_doris. Both JDBC client implementations probe this variable first and use exact identity matching. If the remote endpoint does not expose the variable, they fall back to the existing version_comment detection so older endpoints remain compatible.

The compatibility adjustment for existing enterprise version comments remains a separate commit, allowing it to be reviewed or applied independently.

Release note

Use a stable server identity when detecting Doris JDBC endpoints, with backward-compatible fallback for older endpoints.

Check List (For Author)

  • Test: Unit Test
    • ./run-fe-ut.sh --run org.apache.doris.datasource.jdbc.client.JdbcMySQLClientTest,org.apache.doris.connector.jdbc.client.JdbcMySQLConnectorClientTest
    • Both test classes passed: 4 tests total, 0 failures, 0 errors, 0 skipped
    • FE build, checkstyle, and metadata funnel checks passed as part of the test command
  • Behavior changed: Yes. JDBC Catalog prefers the stable server identity and falls back to the existing version comment probe when the identity variable is unavailable.
  • Does this need documentation: No

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: JDBC Catalog can misidentify an enterprise Doris endpoint as MySQL when its version comment omits optional product markers. This skips Doris-specific type mapping and can expose incorrect schemas. Recognize the stable enterprise version prefix in both JDBC client implementations while retaining the existing compatibility rules.

### Release note

Fix JDBC Catalog type mapping for enterprise Doris endpoints whose version comment omits optional product markers.

### Check List (For Author)

- Test: Unit Test
    - Legacy and connector JDBC endpoint detection tests
- Behavior changed: Yes, recognized enterprise endpoints use Doris-specific type mapping
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: JDBC Catalog infers whether a MySQL-compatible endpoint is Doris from version_comment. Product branding and packaging can change that display string and cause Doris-specific types to be mapped as MySQL types. Expose a stable read-only session variable, prefer it in both JDBC client implementations, and retain version_comment as a fallback for older endpoints.

### Release note

Use a stable server identity when detecting Doris JDBC endpoints.

### Check List (For Author)

- Test: Unit Test
    - Legacy and connector JDBC endpoint identity and compatibility detection tests
- Behavior changed: Yes, JDBC Catalog prefers doris_server_identity and falls back to version_comment for older endpoints
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 27218 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c8b014225eefb04df63bb99aadd1b4d4ca646c52, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17615	3878	3791	3791
q2	2165	353	308	308
q3	10074	1342	817	817
q4	4692	469	346	346
q5	7471	826	533	533
q6	177	165	136	136
q7	742	783	611	611
q8	9300	1349	1400	1349
q9	5361	4162	4133	4133
q10	6820	1328	997	997
q11	425	262	233	233
q12	631	407	292	292
q13	18115	2586	1991	1991
q14	259	258	244	244
q15	q16	726	709	663	663
q17	1847	1131	1020	1020
q18	6478	5593	5540	5540
q19	1164	1200	980	980
q20	490	380	260	260
q21	5397	2923	2674	2674
q22	425	344	300	300
Total cold run time: 100374 ms
Total hot run time: 27218 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4100	4017	4013	4013
q2	716	556	517	517
q3	4466	4845	4301	4301
q4	2192	2306	1428	1428
q5	4195	4079	4059	4059
q6	216	169	125	125
q7	1689	1558	1444	1444
q8	2188	1857	2282	1857
q9	7487	7342	7262	7262
q10	3712	3609	3188	3188
q11	565	398	365	365
q12	715	717	525	525
q13	2405	2736	2182	2182
q14	302	309	280	280
q15	q16	715	715	627	627
q17	7913	7165	7094	7094
q18	11967	11036	11745	11036
q19	1202	1073	1085	1073
q20	2234	2204	1963	1963
q21	5252	4369	4372	4369
q22	490	438	383	383
Total cold run time: 64721 ms
Total hot run time: 58091 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 152387 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c8b014225eefb04df63bb99aadd1b4d4ca646c52, data reload: false

query5	4332	585	451	451
query6	435	212	198	198
query7	4826	537	303	303
query8	326	173	161	161
query9	8799	3952	3901	3901
query10	465	309	254	254
query11	5791	3528	3234	3234
query12	149	88	83	83
query13	1294	566	410	410
query14	6499	4560	4269	4269
query14_1	3998	4009	3968	3968
query15	205	196	182	182
query16	972	481	447	447
query17	914	686	555	555
query18	2450	453	338	338
query19	203	179	140	140
query20	86	78	82	78
query21	218	130	112	112
query22	12973	12883	12803	12803
query23	13922	13151	12461	12461
query23_1	12675	12603	12778	12603
query24	7239	1114	649	649
query24_1	714	681	707	681
query25	571	455	377	377
query26	1307	303	167	167
query27	2651	556	341	341
query28	4557	1957	2003	1957
query29	1611	719	519	519
query30	295	222	179	179
query31	900	754	640	640
query32	138	93	89	89
query33	520	311	247	247
query34	1190	1126	627	627
query35	759	740	659	659
query36	807	808	700	700
query37	147	106	95	95
query38	1830	1779	1705	1705
query39	690	687	659	659
query39_1	677	639	629	629
query40	231	124	101	101
query41	71	69	69	69
query42	99	95	93	93
query43	337	344	303	303
query44	1348	709	699	699
query45	183	176	168	168
query46	1046	1182	733	733
query47	1473	1516	1404	1404
query48	405	399	289	289
query49	584	441	302	302
query50	941	346	261	261
query51	10184	10421	10354	10354
query52	87	88	75	75
query53	238	248	174	174
query54	256	215	177	177
query55	80	75	69	69
query56	232	202	215	202
query57	1464	1444	1434	1434
query58	289	260	256	256
query59	2019	2074	1852	1852
query60	270	241	228	228
query61	146	148	147	147
query62	407	315	268	268
query63	213	173	174	173
query64	2791	979	811	811
query65	3480	3415	3411	3411
query66	1786	416	307	307
query67	19958	19846	19819	19819
query68	3197	1495	903	903
query69	403	300	251	251
query70	860	796	811	796
query71	303	230	216	216
query72	2672	2572	2242	2242
query73	835	770	441	441
query74	4625	4473	4285	4285
query75	2290	2270	1944	1944
query76	2347	1084	763	763
query77	357	373	299	299
query78	9062	9016	8405	8405
query79	1398	1203	762	762
query80	774	461	368	368
query81	568	321	275	275
query82	603	165	129	129
query83	307	214	187	187
query84	322	146	112	112
query85	883	464	369	369
query86	369	241	229	229
query87	1975	1968	1833	1833
query88	3614	2723	2679	2679
query89	373	286	244	244
query90	1845	177	178	177
query91	167	157	125	125
query92	102	86	88	86
query93	1404	1461	869	869
query94	601	352	312	312
query95	645	365	419	365
query96	1098	781	334	334
query97	2457	2434	2319	2319
query98	161	148	143	143
query99	714	718	621	621
Total cold run time: 235739 ms
Total hot run time: 152387 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 23.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c8b014225eefb04df63bb99aadd1b4d4ca646c52, data reload: false

query1	0.01	0.01	0.00
query2	0.09	0.04	0.05
query3	0.26	0.13	0.14
query4	1.60	0.15	0.14
query5	0.25	0.21	0.22
query6	1.16	0.93	0.93
query7	0.04	0.00	0.00
query8	0.05	0.04	0.04
query9	0.39	0.34	0.33
query10	0.57	0.53	0.56
query11	0.20	0.14	0.14
query12	0.17	0.15	0.14
query13	0.45	0.44	0.46
query14	0.96	0.96	0.93
query15	0.59	0.58	0.58
query16	0.31	0.31	0.33
query17	1.07	1.08	1.08
query18	0.23	0.20	0.20
query19	2.06	1.95	1.86
query20	0.02	0.02	0.02
query21	15.47	0.19	0.12
query22	4.99	0.06	0.05
query23	16.11	0.31	0.13
query24	2.98	0.43	0.32
query25	0.10	0.05	0.05
query26	0.74	0.20	0.16
query27	0.05	0.04	0.03
query28	3.50	0.76	0.35
query29	12.63	4.12	3.22
query30	0.28	0.15	0.15
query31	2.77	0.55	0.30
query32	3.23	0.59	0.49
query33	3.22	3.13	3.31
query34	15.51	3.90	3.25
query35	3.23	3.22	3.21
query36	0.55	0.42	0.41
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.03	0.03
query40	0.17	0.14	0.15
query41	0.09	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 96.35 s
Total hot run time: 23.76 s

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Use the generic read-only session variable name `server_identity` for stable JDBC server identification and update both JDBC client implementations to query it.

### Release note

Rename the read-only JDBC server identity variable to `server_identity`.

### Check List (For Author)

- Test: Unit Test
    - JdbcMySQLClientTest
    - JdbcMySQLConnectorClientTest
- Behavior changed: Yes. The read-only identity variable is exposed as `server_identity`.
- Does this need documentation: No
@suxiaogang223

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 27523 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8f714fa11d736848b171e5a259a1b80a3b013d22, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17629	3778	3842	3778
q2	2165	374	316	316
q3	10098	1369	789	789
q4	4676	484	348	348
q5	7520	825	549	549
q6	175	170	136	136
q7	731	791	608	608
q8	9356	1591	1513	1513
q9	5366	4177	4154	4154
q10	6809	1319	1015	1015
q11	432	271	249	249
q12	635	416	304	304
q13	18055	2635	2001	2001
q14	266	265	241	241
q15	q16	733	715	667	667
q17	1735	1100	1048	1048
q18	6498	5602	5541	5541
q19	1149	1227	1026	1026
q20	460	399	273	273
q21	5469	2926	2667	2667
q22	423	344	300	300
Total cold run time: 100380 ms
Total hot run time: 27523 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4124	4023	4000	4000
q2	706	572	507	507
q3	4429	4833	4307	4307
q4	2174	2318	1420	1420
q5	4199	4115	4121	4115
q6	227	173	127	127
q7	1717	1597	1471	1471
q8	2244	2147	2250	2147
q9	7567	7468	7560	7468
q10	3687	3608	3172	3172
q11	561	437	363	363
q12	736	729	511	511
q13	2507	2763	2166	2166
q14	289	314	280	280
q15	q16	693	694	648	648
q17	7905	7076	7088	7076
q18	11875	11090	11859	11090
q19	1168	1096	1065	1065
q20	2287	2247	1958	1958
q21	5224	4381	4606	4381
q22	540	468	413	413
Total cold run time: 64859 ms
Total hot run time: 58685 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 151711 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8f714fa11d736848b171e5a259a1b80a3b013d22, data reload: false

query5	4315	594	445	445
query6	431	206	192	192
query7	4833	558	284	284
query8	322	179	160	160
query9	8813	3924	3963	3924
query10	475	311	260	260
query11	5727	3553	3236	3236
query12	152	94	86	86
query13	1264	642	407	407
query14	6550	4516	4211	4211
query14_1	3978	3931	3958	3931
query15	205	196	173	173
query16	982	488	446	446
query17	924	665	552	552
query18	2446	477	333	333
query19	199	179	145	145
query20	85	80	82	80
query21	221	132	114	114
query22	13039	12920	12823	12823
query23	13794	12838	12423	12423
query23_1	12413	12592	12392	12392
query24	7365	1061	642	642
query24_1	684	703	685	685
query25	543	421	345	345
query26	1267	294	153	153
query27	2807	553	326	326
query28	4571	1940	1936	1936
query29	1623	694	537	537
query30	313	217	180	180
query31	874	759	636	636
query32	141	94	92	92
query33	510	296	237	237
query34	1208	1129	637	637
query35	699	748	638	638
query36	788	803	721	721
query37	144	101	90	90
query38	1829	1759	1699	1699
query39	697	691	657	657
query39_1	636	673	655	655
query40	223	120	102	102
query41	66	96	62	62
query42	97	92	95	92
query43	330	341	289	289
query44	1360	694	710	694
query45	178	173	163	163
query46	1107	1186	739	739
query47	1470	1470	1378	1378
query48	396	422	291	291
query49	582	399	291	291
query50	1008	354	248	248
query51	10458	10365	10681	10365
query52	86	85	73	73
query53	240	249	176	176
query54	257	196	194	194
query55	83	73	68	68
query56	221	205	213	205
query57	1500	1463	1229	1229
query58	268	256	256	256
query59	1978	2049	1858	1858
query60	289	238	223	223
query61	143	146	152	146
query62	398	318	265	265
query63	218	168	177	168
query64	2791	965	791	791
query65	3460	3381	3421	3381
query66	1791	428	312	312
query67	20153	19867	19748	19748
query68	3131	1544	944	944
query69	460	307	258	258
query70	880	833	815	815
query71	287	238	218	218
query72	2706	2612	2193	2193
query73	813	745	431	431
query74	4621	4494	4295	4295
query75	2279	2281	1941	1941
query76	2325	1114	762	762
query77	356	394	298	298
query78	8976	8990	8461	8461
query79	1254	1131	733	733
query80	510	447	360	360
query81	520	319	279	279
query82	275	168	124	124
query83	213	221	201	201
query84	295	144	112	112
query85	796	463	382	382
query86	301	240	235	235
query87	1984	1955	1815	1815
query88	3608	2706	2707	2706
query89	329	279	247	247
query90	2109	182	175	175
query91	165	158	126	126
query92	104	90	85	85
query93	1500	1383	870	870
query94	518	337	302	302
query95	650	377	417	377
query96	1049	832	335	335
query97	2405	2414	2323	2323
query98	156	150	145	145
query99	710	725	627	627
Total cold run time: 234889 ms
Total hot run time: 151711 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.21 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8f714fa11d736848b171e5a259a1b80a3b013d22, data reload: false

query1	0.01	0.00	0.01
query2	0.09	0.06	0.05
query3	0.26	0.14	0.14
query4	1.61	0.14	0.14
query5	0.24	0.23	0.22
query6	1.16	0.96	0.92
query7	0.04	0.00	0.00
query8	0.05	0.04	0.03
query9	0.39	0.34	0.34
query10	0.58	0.58	0.58
query11	0.21	0.14	0.15
query12	0.18	0.14	0.14
query13	0.48	0.47	0.48
query14	0.94	0.95	0.95
query15	0.60	0.58	0.60
query16	0.31	0.33	0.32
query17	1.12	1.12	1.08
query18	0.22	0.19	0.20
query19	1.96	1.98	1.94
query20	0.02	0.01	0.01
query21	15.48	0.21	0.13
query22	4.82	0.05	0.05
query23	16.13	0.33	0.12
query24	2.94	0.44	0.35
query25	0.10	0.06	0.03
query26	0.73	0.21	0.15
query27	0.04	0.04	0.04
query28	3.56	0.80	0.37
query29	12.52	4.00	3.19
query30	0.28	0.16	0.15
query31	2.78	0.55	0.32
query32	3.22	0.59	0.49
query33	3.19	3.24	3.28
query34	15.58	3.93	3.32
query35	3.22	3.23	3.27
query36	0.56	0.44	0.43
query37	0.09	0.07	0.06
query38	0.06	0.03	0.03
query39	0.04	0.03	0.03
query40	0.17	0.14	0.15
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 96.14 s
Total hot run time: 24.21 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants