com.alibaba.alink.common.io

Class HiveDB

java.lang.Object

com.alibaba.alink.common.io.BaseDB

com.alibaba.alink.common.io.HiveDB

All Implemented Interfaces:

Serializable

public class HiveDB
extends BaseDB
implements Serializable

Some pitfalls: 1) datanucleus-api-jdo-4.2.4.jar and datanucleus-core-4.1.17.jar plugin.xml conflict 2) flink config should: classloader.resolve-order: parent-first 3) Hive client version should match with the target Hive to the level of middle version number (like 2.3.0) Otherwise, the target Hive's schema version may be corrupted by the hive client (changed to 2.0.0 for example)

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
HiveDB(Params params)
HiveDB(String hiveConfDir, String hiveVersion, String dbName)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.flink.api.common.io.RichOutputFormat	createFormat(String tableName, org.apache.flink.table.api.TableSchema schema)
void	createTable(String tableName, org.apache.flink.table.api.TableSchema schema, Params parameter)
void	dropTable(String tableName)
void	execute(String sql)
org.apache.flink.table.api.Table	getBatchTable(String tableName, Params parameter, Long sessionId)
String[]	getColNames(String tableName)
BaseDbTable	getDbTable(String tableName)
String	getHiveConfDir()
String	getHiveVersion()
String	getName()
List<String>	getPartitionCols(String tableName)
org.apache.flink.table.api.Table	getStreamTable(String tableName, Params parameter, Long sessionId)
org.apache.flink.table.api.TableSchema	getTableSchema(String tableName)
boolean	hasColumn(String table, String column)
boolean	hasTable(String table)
List<String>	listTableNames()
void	sinkBatch(String tableName, org.apache.flink.table.api.Table in, Params parameter, Long sessionId)
void	sinkStream(String tableName, org.apache.flink.table.api.Table in, Params parameter, Long sessionId)

Methods inherited from class com.alibaba.alink.common.io.BaseDB

bucketingSinkStream, bucketingSinkStream, createTable, createTable, directRead, getBatchTable, getParams, getStreamTable, initConnector, isDB, of, sinkStream

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HiveDB

public HiveDB(String hiveConfDir,
              String hiveVersion,
              String dbName)

HiveDB

public HiveDB(Params params)

Method Detail

getHiveVersion

public String getHiveVersion()

getHiveConfDir

public String getHiveConfDir()

getName

public String getName()

Specified by:

getName in class BaseDB

listTableNames

public List<String> listTableNames()
                            throws Exception

Specified by:

listTableNames in class BaseDB

Throws:

Exception

execute

public void execute(String sql)
             throws Exception

Specified by:

execute in class BaseDB

Throws:

Exception

createTable

public void createTable(String tableName,
                        org.apache.flink.table.api.TableSchema schema,
                        Params parameter)
                 throws Exception

Specified by:

createTable in class BaseDB

Throws:

Exception

dropTable

public void dropTable(String tableName)
               throws Exception

Specified by:

dropTable in class BaseDB

Throws:

Exception

hasTable

public boolean hasTable(String table)
                 throws Exception

Specified by:

hasTable in class BaseDB

Throws:

Exception

hasColumn

public boolean hasColumn(String table,
                         String column)
                  throws Exception

Specified by:

hasColumn in class BaseDB

Throws:

Exception

getColNames

public String[] getColNames(String tableName)
                     throws Exception

Specified by:

getColNames in class BaseDB

Throws:

Exception

getTableSchema

public org.apache.flink.table.api.TableSchema getTableSchema(String tableName)
                                                      throws Exception

Specified by:

getTableSchema in class BaseDB

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in class BaseDB

Throws:

Exception

getDbTable

public BaseDbTable getDbTable(String tableName)
                       throws Exception

Specified by:

getDbTable in class BaseDB

Throws:

Exception

getPartitionCols

public List<String> getPartitionCols(String tableName)

getBatchTable

public org.apache.flink.table.api.Table getBatchTable(String tableName,
                                                      Params parameter,
                                                      Long sessionId)
                                               throws Exception

Specified by:

getBatchTable in class BaseDB

Throws:

Exception

getStreamTable

public org.apache.flink.table.api.Table getStreamTable(String tableName,
                                                       Params parameter,
                                                       Long sessionId)
                                                throws Exception

Specified by:

getStreamTable in class BaseDB

Throws:

Exception

sinkStream

public void sinkStream(String tableName,
                       org.apache.flink.table.api.Table in,
                       Params parameter,
                       Long sessionId)

Specified by:

sinkStream in class BaseDB

sinkBatch

public void sinkBatch(String tableName,
                      org.apache.flink.table.api.Table in,
                      Params parameter,
                      Long sessionId)

Specified by:

sinkBatch in class BaseDB

createFormat

public org.apache.flink.api.common.io.RichOutputFormat createFormat(String tableName,
                                                                    org.apache.flink.table.api.TableSchema schema)

Specified by:

createFormat in class BaseDB