Override QueryExecution with special debug workflow.
Override QueryExecution with special debug workflow.
Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.
Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.
Right now, it only supports Hive tables and it only updates the size of a Hive table in the Hive metastore.
Fewer partitions to speed up testing.
Fewer partitions to speed up testing.
Sets up the system initially or after a RESET command
Sets up the system initially or after a RESET command
When true, a table created by a Hive CTAS statement (no USING clause) will be converted to a data source table, using the data source set by spark.sql.sources.default.
When true, a table created by a Hive CTAS statement (no USING clause) will be converted to a data source table, using the data source set by spark.sql.sources.default. The table in CTAS statement will be converted when it meets any of the following conditions:
When true, enables an experimental feature where metastore tables that use the parquet SerDe are automatically converted to use the Spark SQL parquet table scan, instead of the Hive SerDe.
When true, enables an experimental feature where metastore tables that use the parquet SerDe are automatically converted to use the Spark SQL parquet table scan, instead of the Hive SerDe.
The location of the hive source code.
The location of the hive source code.
The location of the compiled hive distribution
The location of the compiled hive distribution
Records the UDFs present when the server starts, so we can delete ones that are created by tests.
Records the UDFs present when the server starts, so we can delete ones that are created by tests.
Invalidate and refresh all the cached the metadata of the given table.
Invalidate and refresh all the cached the metadata of the given table. For performance reasons, Spark SQL or the external data source library it uses might cache certain metadata about a table, such as the location of blocks. When those change outside of Spark SQL, users should call this function to invalidate the cache.
Resets the test instance by deleting any tables that have been created.
Resets the test instance by deleting any tables that have been created. TODO: also clear out UDFs, views, etc.
Execute the command using Hive and return the results as a sequence.
Execute the command using Hive and return the results as a sequence. Each element in the sequence is one row.
Runs the specified SQL query using Hive.
Runs the specified SQL query using Hive.
SQLConf and HiveConf contracts:
SQLConf and HiveConf contracts:
1. reuse existing started SessionState if any 2. when the Hive session is first initialized, params in HiveConf will get picked up by the SQLConf. Additionally, any properties set by set() or a SET command inside sql() will be set in the SQLConf *as well as* in the HiveConf.
A list of test tables and the DDL required to initialize them.
A list of test tables and the DDL required to initialize them. A test table is loaded on demand when a query are run against it.
(Since version 1.3.0) use createDataFrame
(Since version 1.3.0) use createDataFrame
(Since version 1.3.0) use createDataFrame
(Since version 1.3.0) use createDataFrame