Skip to content

Commit f717eb2

Browse files
MrRahulSharmaMrRahulSharma
authored
Converting MySQL YEAR type to CDAP Integer type (#388)
1 parent 29754d8 commit f717eb2

File tree

12 files changed

+263
-77
lines changed

12 files changed

+263
-77
lines changed

‎mysql-plugin/docs/Mysql-action.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ port 3306 (Ensure that the driver for MySQL is installed. You can also provide d
6262
otherwise "mysql" will be used), then configure the plugin with:
6363

6464
```
65-
Driver Name: "mariadb"
65+
Driver Name: "mysql"
6666
Database Command: "UPDATE table_name SET price = 20 WHERE ID = 6"
6767
Host: "localhost"
6868
Port: 3306

‎mysql-plugin/docs/Mysql-batchsink.md‎

Lines changed: 32 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -66,38 +66,37 @@ connections.
6666

6767
Data Types Mapping
6868
----------
69-
70-
| MySQL Data Type | CDAP Schema Data Type | Comment |
71-
| ------------------------------ | --------------------- | -------------------------------------------------- |
72-
| BIT | boolean | |
73-
| TINYINT | int | |
74-
| BOOL, BOOLEAN | boolean | |
75-
| SMALLINT | int | |
76-
| MEDIUMINT | double | |
77-
| INT,INTEGER | int | |
78-
| BIGINT | long | |
79-
| FLOAT | float | |
80-
| DOUBLE | double | |
81-
| DECIMAL | decimal | |
82-
| DATE | date | |
83-
| DATETIME | timestamp | |
84-
| TIMESTAMP | timestamp | |
85-
| TIME | time | |
86-
| YEAR | date | |
87-
| CHAR | string | |
88-
| VARCHAR | string | |
89-
| BINARY | bytes | |
90-
| VARBINARY | bytes | |
91-
| TINYBLOB | bytes | |
92-
| TINYTEXT | string | |
93-
| BLOB | bytes | |
94-
| TEXT | string | |
95-
| MEDIUMBLOB | bytes | |
96-
| MEDIUMTEXT | string | |
97-
| LONGBLOB | bytes | |
98-
| LONGTEXT | string | |
99-
| ENUM | string | |
100-
| SET | string | |
69+
| MySQL Data Type | CDAP Schema Data Type | Comment |
70+
| ------------------------------ |-----------------------|---------------------------------------------------------|
71+
| BIT | boolean | |
72+
| TINYINT | int | |
73+
| BOOL, BOOLEAN | boolean | |
74+
| SMALLINT | int | |
75+
| MEDIUMINT | double | |
76+
| INT,INTEGER | int | |
77+
| BIGINT | long | |
78+
| FLOAT | float | |
79+
| DOUBLE | double | |
80+
| DECIMAL | decimal | |
81+
| DATE | date | |
82+
| DATETIME | timestamp | |
83+
| TIMESTAMP | timestamp | |
84+
| TIME | time | |
85+
| YEAR | int | Users can manually set output schema to map it to Date. |
86+
| CHAR | string | |
87+
| VARCHAR | string | |
88+
| BINARY | bytes | |
89+
| VARBINARY | bytes | |
90+
| TINYBLOB | bytes | |
91+
| TINYTEXT | string | |
92+
| BLOB | bytes | |
93+
| TEXT | string | |
94+
| MEDIUMBLOB | bytes | |
95+
| MEDIUMTEXT | string | |
96+
| LONGBLOB | bytes | |
97+
| LONGTEXT | string | |
98+
| ENUM | string | |
99+
| SET | string | |
101100

102101

103102
Example
@@ -108,7 +107,7 @@ driver name for some specific driver, otherwise "mysql" will be used), then conf
108107

109108
```
110109
Reference Name: "snk1"
111-
Driver Name: "mariadb"
110+
Driver Name: "mysql"
112111
Host: "localhost"
113112
Port: 3306
114113
Database: "prod"

‎mysql-plugin/docs/Mysql-batchsource.md‎

Lines changed: 32 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -84,39 +84,37 @@ with the tradeoff of higher memory usage.
8484

8585
Data Types Mapping
8686
----------
87-
88-
| MySQL Data Type | CDAP Schema Data Type | Comment |
89-
| ------------------------------ | --------------------- | -------------------------------------------------- |
90-
| BIT | boolean | |
91-
| TINYINT | int | |
92-
| BOOL, BOOLEAN | boolean | |
93-
| SMALLINT | int | |
94-
| MEDIUMINT | double | |
95-
| INT,INTEGER | int | |
96-
| BIGINT | long | |
97-
| FLOAT | float | |
98-
| DOUBLE | double | |
99-
| DECIMAL | decimal | |
100-
| DATE | date | |
101-
| DATETIME | timestamp | |
102-
| TIMESTAMP | timestamp | |
103-
| TIME | time | |
104-
| YEAR | date | |
105-
| CHAR | string | |
106-
| VARCHAR | string | |
107-
| BINARY | bytes | |
108-
| VARBINARY | bytes | |
109-
| TINYBLOB | bytes | |
110-
| TINYTEXT | string | |
111-
| BLOB | bytes | |
112-
| TEXT | string | |
113-
| MEDIUMBLOB | bytes | |
114-
| MEDIUMTEXT | string | |
115-
| LONGBLOB | bytes | |
116-
| LONGTEXT | string | |
117-
| ENUM | string | |
118-
| SET | string | |
119-
87+
| MySQL Data Type | CDAP Schema Data Type | Comment |
88+
| ------------------------------ | --------------------- |---------------------------------------------------------|
89+
| BIT | boolean | |
90+
| TINYINT | int | |
91+
| BOOL, BOOLEAN | boolean | |
92+
| SMALLINT | int | |
93+
| MEDIUMINT | double | |
94+
| INT,INTEGER | int | |
95+
| BIGINT | long | |
96+
| FLOAT | float | |
97+
| DOUBLE | double | |
98+
| DECIMAL | decimal | |
99+
| DATE | date | |
100+
| DATETIME | timestamp | |
101+
| TIMESTAMP | timestamp | |
102+
| TIME | time | |
103+
| YEAR | int | Users can manually set output schema to map it to Date. |
104+
| CHAR | string | |
105+
| VARCHAR | string | |
106+
| BINARY | bytes | |
107+
| VARBINARY | bytes | |
108+
| TINYBLOB | bytes | |
109+
| TINYTEXT | string | |
110+
| BLOB | bytes | |
111+
| TEXT | string | |
112+
| MEDIUMBLOB | bytes | |
113+
| MEDIUMTEXT | string | |
114+
| LONGBLOB | bytes | |
115+
| LONGTEXT | string | |
116+
| ENUM | string | |
117+
| SET | string | |
120118

121119
Example
122120
------
@@ -127,7 +125,7 @@ driver name for some specific driver, otherwise "mysql" will be used), then con
127125

128126
```
129127
Reference Name: "src1"
130-
Driver Name: "mariadb"
128+
Driver Name: "mysql"
131129
Host: "localhost"
132130
Port: 3306
133131
Database: "prod"

‎mysql-plugin/docs/Mysql-postaction.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ then configure the plugin with:
7777

7878
```
7979
Run Condition: "success"
80-
Driver Name: "maridb"
80+
Driver Name: "mysql"
8181
Query: "delete * from userEvents"
8282
Host: "localhost"
8383
Port: 3306

‎mysql-plugin/src/e2e-test/java/io/cdap/plugin/MysqlClient.java‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616

1717
package io.cdap.plugin;
1818

19+
import com.google.common.base.Strings;
1920
import io.cdap.e2e.utils.PluginPropertyUtils;
2021
import org.junit.Assert;
2122

@@ -103,13 +104,23 @@ public static boolean compareResultSetData(ResultSet rsSource, ResultSet rsTarge
103104
gc.setGregorianChange(new Date(Long.MIN_VALUE));
104105
Timestamp sourceTS = rsSource.getTimestamp(currentColumnCount, gc);
105106
Timestamp targetTS = rsTarget.getTimestamp(currentColumnCount, gc);
106-
Assert.assertTrue(String.format("Different values found for column : %s", columnName),
107-
sourceTS.equals(targetTS));
107+
if (sourceTS == null) {
108+
Assert.assertNull(String.format("Not null TIMESTAMP value found in target for column : %s", columnName),
109+
targetTS);
110+
} else {
111+
Assert.assertEquals(String.format("Different TIMESTAMP values found for column : %s", columnName),
112+
sourceTS, targetTS);
113+
}
108114
} else {
109115
String sourceString = rsSource.getString(currentColumnCount);
110116
String targetString = rsTarget.getString(currentColumnCount);
111-
Assert.assertTrue(String.format("Different values found for column : %s", columnName),
112-
String.valueOf(sourceString).equals(String.valueOf(targetString)));
117+
if (Strings.isNullOrEmpty(sourceString)) {
118+
Assert.assertTrue(String.format("Not null/empty value found in target for column : %s", columnName),
119+
Strings.isNullOrEmpty(targetString));
120+
} else {
121+
Assert.assertEquals(String.format("Different values found for column : %s", columnName),
122+
sourceString, targetString);
123+
}
113124
}
114125
currentColumnCount++;
115126
}

‎mysql-plugin/src/e2e-test/resources/pluginParameters.properties‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,16 +13,16 @@ datatypesColumns=(ID varchar(100) PRIMARY KEY, COL1 bigint(20), COL2 bigint(20)
1313
COL12 enum('A','B','C'), COL13 float, COL14 int(11), COL15 int(10) unsigned, COL16 mediumblob, COL17 mediumtext, \
1414
COL18 longblob, COL19 longtext, COL20 mediumint(9), COL21 mediumint(8) unsigned, COL22 set('X','y','Z'), \
1515
COL23 smallint(6), COL24 smallint(5) unsigned, COL25 text, COL26 time, COL27 timestamp, COL28 tinyblob, \
16-
COL29 tinyint(4), COL30 tinyint(3) unsigned, COL31 tinytext, COL32 varbinary(100), COL33 json)
16+
COL29 tinyint(4), COL30 tinyint(3) unsigned, COL31 tinytext, COL32 varbinary(100), COL33 json, COL34 year)
1717
datatypesColumnsList=(ID,COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8,COL9,COL10,COL11,COL12,COL13,COL14,COL15,COL16,COL17,\
18-
COL18,COL19,COL20,COL21,COL22,COL23,COL24,COL25,COL26,COL27,COL28,COL29,COL30,COL31,COL32,COL33)
18+
COL18,COL19,COL20,COL21,COL22,COL23,COL24,COL25,COL26,COL27,COL28,COL29,COL30,COL31,COL32,COL33,COL34)
1919
datatypesValues=VALUES ('User1',1000000000000000000,1000000000000000000,1,1,\
2020
HEX('27486920546869732069732061206C6F6E6720746578742E27'),1,'A','2023-01-01','2023-01-01 00:00:00',1234,\
2121
1234.5678,'A',22.0,-1234,1234,HEX('27486920546869732069732061206C6F6E6720746578742E27'),\
2222
'This is a test message',HEX('27486920546869732069732061206C6F6E6720746578742E27'),\
2323
'This is a test message\n\n',-1234,1234,'X',-1234,1234,'This is a test message','00:00:00','2023-01-01 00:00:00',\
2424
HEX('27486920546869732069732061206C6F6E6720746578742E27'),-100,100,'This is a test message',1,\
25-
'{"key1": "value1", "key2": "value2"}')
25+
'{"key1": "value1", "key2": "value2"}',2023)
2626
datatypesSchema=[{"key":"ID","value":"string"},{"key":"COL1","value":"long"},{"key":"COL2","value":"decimal"},\
2727
{"key":"COL3","value":"bytes"},{"key":"COL4","value":"boolean"},{"key":"COL5","value":"bytes"},\
2828
{"key":"COL6","value":"boolean"},{"key":"COL7","value":"string"},{"key":"COL8","value":"date"},\
@@ -34,4 +34,4 @@ datatypesSchema=[{"key":"ID","value":"string"},{"key":"COL1","value":"long"},{"k
3434
{"key":"COL24","value":"int"},{"key":"COL25","value":"string"},{"key":"COL26","value":"time"},\
3535
{"key":"COL27","value":"timestamp"},{"key":"COL28","value":"bytes"},{"key":"COL29","value":"int"},\
3636
{"key":"COL30","value":"int"},{"key":"COL31","value":"string"},{"key":"COL32","value":"bytes"},\
37-
{"key":"COL33","value":"string"}]
37+
{"key":"COL33","value":"string"},{"key":"COL34","value":"int"}]

0 commit comments

Comments
 (0)