Flink?????????Table??SQL API
???????????? ???????[ 2017/3/31 11:16:32 ] ????????????? SQL
????Flink????????????DataStream/DataSetAPI???????????????????????????е?????????????????Table&SQLAPI???
????????TableAPI?????????API????SQL??API?????????????????????????????????????????????????????????????????????????????θ?Ч??????????????????DataStream???д????Function????????????????????????SQL??????“???????”???????????????????SQL??????????????????????????????????????????????????????Flink??TableAPI??SQLAPI????????80%??????????????????????????TableAPI??????????Table&SQLAPI??
????Table&SQLAPI???????????????????????????????API??Flink??runtime????????????Flink?????????????????????????У??????Flink????????????????????????Flink????????????????API??DataSet??DataStream???????runtime?????????????????????????????????????????顣??????????runtime?????????????????????д??????????????????ApacheBeam???????????????Table&SQLAPI????????API??????????????????????????????????????????????????????????????????в???????????Table&SQLAPI?????????????????????????????????????????????????????????
???????????
????TableAPI???Flink0.9??Flink0.9??????????????汾???????????TableAPI??Gelly??FlinkML??????????汾???????Flink0.9???????2015??6??????????????Flink0.9??????????????SQL??????ü?????????????????????????????????TableAPI???????????SQL??????????????????????????á???????????Hadoop??????????д???????ν“SQL-on-Hadoop”?????????????ApacheHive??ApacheDrill??ApacheImpala??”SQL-on-Flink”?????????????Hadoop??????????????????
??????????????Flink0.9?????????ζ??????????????????ɡ???????????????????????SQL???????????????·????????棬?????SQL????????????2015???°???Timo???????????dataArtisans???????TableAPI?????????2016???????????????????????????????????????????TableAPI????????????????????????????????????Flink????????Apache?л?????????????Flink?л???????????TableAPIī?????????????????????????TableAPI?????????????????????????????????????????TableAPI??????????????????????????????????????????????????????FlinkCommitter??
????TableAPI&SQL????????
?????????????????TableAPI&SQL?????????????????????????????????????????????????
????????????????????????TableAPI?????????????????????????????room?????location????????????????漰????????window??event-time???
valsensorData:DataStream[(String??Long??Double)]=???
//convertDataSetintoTable
valsensorTable:Table=sensorData
.toTable(tableEnv??'location??'time??'tempF)
//definequeryonTable
valavgTempCTable:Table=sensorTable
.window(Tumbleover1.dayon'rowtimeas'w)
.groupBy('location??'w)
.select('w.startas'day??'location??(('tempF.avg-32)*0.556)as'avgTempC)
.where('locationlike"room%")
????????????????????????SQL??????
valsensorData:DataStream[(String??Long??Double)]=???
//registerDataStream
tableEnv.registerDataStream("sensorData"??sensorData??'location??’time??'tempF)
//queryregisteredTable
valavgTempCTable:Table=tableEnv.sql("""
SELECTFLOOR(rowtime()TODAY)ASday??location??
AVG((tempF-32)*0.556)ASavgTempC
FROMsensorData
WHERElocationLIKE'room%'
GROUPBYlocation??FLOOR(rowtime()TODAY)""")
????TableAPI&SQL???
????Flink?????????????Spark?????????????(SparkCatalyst)???????SQLУ?顢SQL???????SQL?????????ApacheCalcite??Calcite????????????????????????????ApacheHive??ApacheDrill??ApacheKylin??Cascading??Calcite???μ????д????????λ????????????
?????μ????У????????????????????????????Calcite?????SQLquery????Calcite??????????SQL???????????????????Calcite????????????????е?LogicalPlan??????????TableAPI???????????TableAPI????????????????Calcite????RelBuilder????Calcite???????????
?????????????????????????TableAPI??SQL????????????£???????????FlinkTableNodes????????????CalciteLogicalNodes?????????????????LogicalPlan?????????
??????????????????Calcite???????????????????ЩLogicalPlan?????????л???????????ò???????????Flink?????????????????????????????????????????????????????Calcite????????????????????????????????????????????LogicalNode????FlinkNode???????????????????????????е??????裬???????趼????Calcite???????Ρ??????DataStreamPlan???????ν?????????DataStream/DataSet???????????????????????DataStream/DataSetNode????????????CodeGen????????????е?DataStream/DataSet????
??????????????TableAPI&SQL?к???????????????????????????ú???????????CodeGen???????Function???????????SparkSQL??????????CodeGen????Function???????????????????????????????????TaskManager?????У?????????????Janino?????????????????С?
????TableAPI&SQL???
??????TableAPI?????????????????????????Selection??Projection??Union?????Window????????????????????????????????????SQL???????Calcite?????汾?в????Window??????????FlinkSQL???????Window??????????TableAPI??SQL???????UDF??UDTF??UDAF(??????)??
????TableAPI&SQLδ??
????1??DynamicTables
????DynamicTable?????????????????????е????????仯???μ??Flink???Stream<–>DynamicTable?????????????????????????????Retraction??????л?????????????д?????????????
????2??Joins
?????????????????????Join???????????Join??
????3??SQL?????
??????SQL??????????Java/Scala?????????е???????SQL????÷????δ????????SQL??????????SQL?????????????
????4?????ж?????
??????TableAPI&SQL????????ò??ж????????TableAPI???????????????
?????????????Flink??Table&SQLAPI???????????????????????????????????????飬????SQL??Window???????????????????????????????????????SQL??и?????????????????????????????????????????SQL??????????????????????Beam???????????????????δ?????Table&SQLAPI??????????????????????????????????????API?????????????BeamTable??BeamSQL?????????????????????????Table&SQLAPI?????????????????????????????????????·???????TableAPI?????????????
??????
???·???
??????????????????
2023/3/23 14:23:39???д?ò??????????
2023/3/22 16:17:39????????????????????Щ??
2022/6/14 16:14:27??????????????????????????
2021/10/18 15:37:44???????????????
2021/9/17 15:19:29???·???????·
2021/9/14 15:42:25?????????????
2021/5/28 17:25:47??????APP??????????
2021/5/8 17:01:11