Skip to content

Commit 63395f7

Browse files
elasticsearch6维表部分
1 parent d5059c8 commit 63395f7

File tree

13 files changed

+796
-643
lines changed

13 files changed

+796
-643
lines changed

docs/elasticsearch6Side

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
2+
## 1.格式:
3+
```
4+
CREATE TABLE tableName(
5+
colName cloType,
6+
...
7+
PRIMARY KEY(keyInfo),
8+
PERIOD FOR SYSTEM_TIME
9+
)WITH(
10+
type='elasticsearch6',
11+
address ='ip:port[,ip:port]',
12+
cluster='clusterName',
13+
estype ='esType',
14+
index ='index',
15+
authMesh='true',
16+
userName='dbUserName',
17+
password='dbPwd',
18+
cache ='LRU',
19+
cacheSize ='10000',
20+
cacheTTLMs ='60000',
21+
parallelism ='1',
22+
partitionedJoin='false'
23+
);
24+
```
25+
26+
# 2.支持版本
27+
mysql-5.6.35
28+
29+
## 3.表结构定义
30+
31+
|参数名称|含义|
32+
|----|---|
33+
| tableName | mysql表名称|
34+
| colName | 列名称|
35+
| colType | 列类型 [colType支持的类型](colType.md)|
36+
| PERIOD FOR SYSTEM_TIME | 关键字表明该定义的表为维表信息|
37+
| PRIMARY KEY(keyInfo) | 维表主键定义;多个列之间用逗号隔开|
38+
39+
## 4.参数
40+
41+
|参数名称|含义|是否必填|默认值|
42+
|----|---|---|----|
43+
type|表明 输出表类型[mysq|hbase|elasticsearch6]|是||
44+
|address | 连接ES Transport地址(tcp地址)|是||
45+
|cluster | ES 集群名称 |是||
46+
|index | 选择的ES上的index名称|否||
47+
|esType | 选择ES上的type名称|否||
48+
|authMesh | 是否进行用户名密码认证 | 否 | false|
49+
|userName | 用户名 | 否,authMesh='true'时为必填 ||
50+
|password | 密码 | 否,authMesh='true'时为必填 ||
51+
| cache | 维表缓存策略(NONE/LRU)|否|NONE|
52+
| partitionedJoin | 是否在維表join之前先根据 設定的key 做一次keyby操作(可以減少维表的数据缓存量)|否|false|
53+
|parallelism | 并行度设置|否|1|
54+
55+
----------
56+
> 缓存策略
57+
* NONE: 不做内存缓存
58+
* LRU:
59+
* cacheSize: 缓存的条目数量
60+
* cacheTTLMs:缓存的过期时间(ms)
61+
* cacheMode: (unordered|ordered)异步加载是有序还是无序,默认有序。
62+
* asyncCapacity:异步请求容量,默认1000
63+
* asyncTimeout:异步请求超时时间,默认10000毫秒
64+
65+
## 5.样例
66+
```
67+
create table sideTable(
68+
channel varchar,
69+
xccount int,
70+
PRIMARY KEY(channel),
71+
PERIOD FOR SYSTEM_TIME
72+
)WITH(
73+
type ='elasticsearch',
74+
address ='172.16.10.47:9500',
75+
cluster='es_47_menghan',
76+
estype ='type1',
77+
index ='xc_es_test',
78+
authMesh='true',
79+
userName='dtstack',
80+
password='abc123',
81+
cache ='LRU',
82+
cacheSize ='10000',
83+
cacheTTLMs ='60000',
84+
cacheMode='unordered',
85+
asyncCapacity='1000',
86+
asyncTimeout='10000'
87+
parallelism ='1',
88+
partitionedJoin='false'
89+
);
90+
91+
92+
```
93+
94+

elasticsearch6/elasticsearch6-side/elasticsearch6-all-side/pom.xml

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,6 @@
3232
<version>${sql.side.elasticsearch6.core.version}</version>
3333
</dependency>
3434

35-
<dependency>
36-
<groupId>junit</groupId>
37-
<artifactId>junit</artifactId>
38-
<version>3.8.1</version>
39-
<scope>test</scope>
40-
</dependency>
4135
<dependency>
4236
<groupId>junit</groupId>
4337
<artifactId>junit</artifactId>

0 commit comments

Comments
 (0)