Skip to content

Commit 89c9235

Browse files
simenliuxingsimenliuxing
committed
Merge remote-tracking branch 'origin/v1.8.0_dev_sqlserver' into v1.8.0_dev
# Conflicts: # README.md
2 parents f46e6a0 + 33a9d61 commit 89c9235

4 files changed

Lines changed: 137 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@
2727

2828
# 已支持
2929
* 源表：kafka 0.9、0.10、0.11、1.x版本
30-
* 维表：mysql, SQlServer,oracle, hbase, mongo, redis, cassandra, serversocket, kudu, postgresql, clickhouse, impala, db2
31-
* 结果表：mysql, SQlServer, oracle, hbase, elasticsearch5.x, mongo, redis, cassandra, console, kudu, postgresql, clickhouse, impala, db2
30+
* 维表：mysql, SQlServer,oracle, hbase, mongo, redis, cassandra, serversocket, kudu, postgresql, clickhouse, impala, db2, sqlserver
31+
* 结果表：mysql, SQlServer, oracle, hbase, elasticsearch5.x, mongo, redis, cassandra, console, kudu, postgresql, clickhouse, impala, db2, sqlserver
3232

3333
# 后续开发计划
3434
* 维表快照
@@ -188,6 +188,7 @@ sh submit.sh -sql D:\sideSql.txt -name xctest -remoteSqlPluginPath /opt/dtstack
188188
* [clickhouse 结果表插件](docs/clickhouseSink.md)
189189
* [impala 结果表插件](docs/impalaSink.md)
190190
* [db2 结果表插件](docs/db2Sink.md)
191+
* [sqlserver 结果表插件](docs/sqlserverSink.md)
191192

192193
### 2.3 维表插件
193194
* [hbase 维表插件](docs/hbaseSide.md)
@@ -201,6 +202,7 @@ sh submit.sh -sql D:\sideSql.txt -name xctest -remoteSqlPluginPath /opt/dtstack
201202
* [clickhouse 维表插件](docs/clickhouseSide.md)
202203
* [impala 维表插件](docs/impalaSide.md)
203204
* [db2 维表插件](docs/db2Side.md)
205+
* [sqlserver 维表插件](docs/sqlserverSide.md)
204206

205207
## 3 性能指标(新增)
206208

‎docs/sqlserverSide.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
2+
## 1.格式：
3+
```
4+
CREATE TABLE tableName(
5+
colName cloType,
6+
...
7+
PRIMARY KEY(keyInfo),
8+
PERIOD FOR SYSTEM_TIME
9+
)WITH(
10+
type='sqlserver',
11+
url='jdbcUrl',
12+
userName='dbUserName',
13+
password='dbPwd',
14+
tableName='tableName',
15+
cache ='LRU',
16+
cacheSize ='10000',
17+
cacheTTLMs ='60000',
18+
parallelism ='1',
19+
partitionedJoin='false'
20+
);
21+
```
22+
23+
# 2.支持版本
24+
mysql-5.6.35
25+
26+
## 3.表结构定义
27+
28+
|参数名称|含义|
29+
|----|---|
30+
| tableName | sqlserver表名称|
31+
| colName | 列名称|
32+
| colType | 列类型 [colType支持的类型](colType.md)|
33+
| PERIOD FOR SYSTEM_TIME | 关键字表明该定义的表为维表信息|
34+
| PRIMARY KEY(keyInfo) | 维表主键定义;多个列之间用逗号隔开|
35+
36+
## 4.参数
37+
38+
|参数名称|含义|是否必填|默认值|
39+
|----|---|---|----|
40+
| type | 表明维表的类型 sqlserver |是||
41+
| url | 连接mysql数据库 jdbcUrl |是||
42+
| userName | sqlserver连接用户名 |是||
43+
| password | sqlserver连接密码|是||
44+
| tableName | sqlserver表名称|是||
45+
| tableName | sqlserver 的表名称|是||
46+
| cache | 维表缓存策略(NONE/LRU)|否|NONE|
47+
| partitionedJoin | 是否在維表join之前先根据 設定的key 做一次keyby操作(可以減少维表的数据缓存量)|否|false|
48+
49+
----------
50+
> 缓存策略
51+
* NONE: 不做内存缓存
52+
* LRU:
53+
* cacheSize: 缓存的条目数量
54+
* cacheTTLMs:缓存的过期时间(ms)
55+
56+
## 5.样例
57+
```
58+
create table sideTable(
59+
channel varchar,
60+
xccount int,
61+
PRIMARY KEY(channel),
62+
PERIOD FOR SYSTEM_TIME
63+
)WITH(
64+
type='sqlserver',
65+
url='jdbc:jtds:sqlserver://172.16.8.104:1433;DatabaseName=mytest',
66+
userName='dtstack',
67+
password='abc123',
68+
tableName='sidetest',
69+
cache ='LRU',
70+
cacheSize ='10000',
71+
cacheTTLMs ='60000',
72+
parallelism ='1',
73+
partitionedJoin='false'
74+
);
75+
76+
77+
```
78+
79+

‎docs/sqlserverSink.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
## 1.格式：
2+
```
3+
CREATE TABLE tableName(
4+
colName colType,
5+
...
6+
colNameX colType
7+
)WITH(
8+
type ='sqlserver',
9+
url ='jdbcUrl',
10+
userName ='userName',
11+
password ='pwd',
12+
tableName ='tableName',
13+
parallelism ='parllNum'
14+
);
15+
16+
```
17+
18+
## 2.支持版本
19+
sqlserver-5.6.35
20+
21+
## 3.表结构定义
22+
23+
|参数名称|含义|
24+
|----|---|
25+
| tableName| sqlserver表名称|
26+
| colName | 列名称|
27+
| colType | 列类型 [colType支持的类型](colType.md)|
28+
29+
## 4.参数：
30+
31+
|参数名称|含义|是否必填|默认值|
32+
|----|----|----|----|
33+
|type |表名 输出表类型[mysq|hbase|elasticsearch]|是||
34+
|url | 连接sqlserver数据库 jdbcUrl |是||
35+
|userName | sqlserver连接用户名 |是||
36+
| password | sqlserver连接密码|是||
37+
| tableName | sqlserver表名称|是||
38+
| parallelism | 并行度设置|否|1|
39+
40+
## 5.样例：
41+
```
42+
CREATE TABLE MyResult(
43+
channel VARCHAR,
44+
pv VARCHAR
45+
)WITH(
46+
type ='sqlserver',
47+
url ='jdbc:jtds:sqlserver://172.16.8.104:1433;DatabaseName=mytest',
48+
userName ='dtstack',
49+
password ='abc123',
50+
tableName ='pv2',
51+
parallelism ='1'
52+
)
53+
```

‎sqlserver/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
<artifactId>jtds</artifactId>
3838
<version>${jtds.version}</version>
3939
</dependency>
40+
4041
</dependencies>
4142

4243
</project>

0 commit comments

Comments
 (0)