Caches the specified table or view in-memory.
Caches the specified table or view in-memory.
is either a qualified or unqualified name that designates a table/view. If no database identifier is provided, it refers to a temporary view or a table/view in the current database.
2.0.0
Removes all cached tables or views from the in-memory cache.
Removes all cached tables or views from the in-memory cache.
2.0.0
:: Experimental :: (Scala-specific) Creates a table based on the dataset in a data source, a schema and a set of options.
:: Experimental :: (Scala-specific) Creates a table based on the dataset in a data source, a schema and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
2.2.0
:: Experimental :: (Scala-specific) Creates a table based on the dataset in a data source and a set of options.
:: Experimental :: (Scala-specific) Creates a table based on the dataset in a data source and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
2.2.0
:: Experimental :: Creates a table from the given path and returns the corresponding DataFrame.
:: Experimental :: Creates a table from the given path and returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
2.2.0
:: Experimental :: Creates a table from the given path and returns the corresponding DataFrame.
:: Experimental :: Creates a table from the given path and returns the corresponding DataFrame. It will use the default data source configured by spark.sql.sources.default.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
2.2.0
:: Experimental :: Create a table based on the dataset in a data source, a schema and a set of options.
:: Experimental :: Create a table based on the dataset in a data source, a schema and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
2.2.0
:: Experimental :: Creates a table based on the dataset in a data source and a set of options.
:: Experimental :: Creates a table based on the dataset in a data source and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
2.2.0
Returns the current default database in this session.
Returns the current default database in this session.
Checks if the database with the specified name exists.
Checks if the database with the specified name exists.
Drops the global temporary view with the given view name in the catalog.
Drops the global temporary view with the given view name in the catalog. If the view has been cached/persisted before, it's also unpersisted.
the identifier of the global temporary view to be dropped.
true if the view is dropped successfully, false otherwise.
2.1.0
Drops the local temporary view with the given view name in the catalog.
Drops the local temporary view with the given view name in the catalog. If the view has been cached/persisted before, it's also unpersisted.
the identifier of the temporary view to be dropped.
true if the view is dropped successfully, false otherwise.
2.0.0
Checks if the function with the specified name exists in the specified database.
Checks if the function with the specified name exists in the specified database.
is a name that designates a database.
is an unqualified name that designates a function.
Checks if the function with the specified name exists.
Checks if the function with the specified name exists. This can either be a temporary function or a function.
is either a qualified or unqualified name that designates a function. If no database identifier is provided, it refers to a function in the current database.
Gets the database with the specified name.
Gets the database with the specified name. This throws an AnalysisException
when no
Database
can be found.
Gets the function with the specified name.
Gets the function with the specified name. This returns None
when no Function
can be
found.
is a name that designates a database.
is an unqualified name that designates a function in the specified database
Gets the function with the specified name.
Gets the function with the specified name. This function can be a temporary function or a
function. This throws an AnalysisException
when no Function
can be found.
is either a qualified or unqualified name that designates a function. If no database identifier is provided, it refers to a temporary function or a function in the current database.
Gets the table or view with the specified name in the specified database.
Gets the table or view with the specified name in the specified database. This throws an
AnalysisException
when no Table
can be found.
Gets the table or view with the specified name.
Gets the table or view with the specified name. This table can be a temporary view or a
table/view. This throws an AnalysisException
when no Table
can be found.
is either a qualified or unqualified name that designates a table/view. If no database identifier is provided, it refers to a table/view in the current database.
Returns true if the Dataset is currently cached in-memory.
Returns true if the Dataset is currently cached in-memory.
2.0.0
Returns true if the table or view is currently cached in-memory.
Returns true if the table or view is currently cached in-memory.
is either a qualified or unqualified name that designates a table/view. If no database identifier is provided, it refers to a temporary view or a table/view in the current database.
2.0.0
Returns a list of columns for the given table/view or temporary view in the specified database.
Returns a list of columns for the given table/view or temporary view in the specified database.
is a name that designates a database.
is an unqualified name that designates a table/view.
Returns a list of columns for the given table/view or temporary view.
Returns a list of columns for the given table/view or temporary view.
is either a qualified or unqualified name that designates a table/view. If no database identifier is provided, it refers to a temporary view or a table/view in the current database.
Returns a list of databases available across all sessions.
Returns a list of databases available across all sessions.
Returns a list of functions registered in the specified database.
Returns a list of functions registered in the specified database. This includes all temporary functions
Returns a list of functions registered in the current database.
Returns a list of functions registered in the current database. This includes all temporary functions
Returns a list of tables in the specified database.
Returns a list of tables in the specified database. This includes all temporary tables.
Returns a list of tables in the current database.
Returns a list of tables in the current database. This includes all temporary tables.
Recovers all the partitions in the directory of a table and update the catalog.
Recovers all the partitions in the directory of a table and update the catalog. Only works with a partitioned table, and not a temporary view.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
2.1.1
Refreshes the cache entry and the associated metadata for all Dataset (if any), that contain the given data source path.
Refreshes the cache entry and the associated metadata for all Dataset (if any), that contain the given data source path. Path matching is by prefix, i.e. "/" would invalidate everything that is cached.
2.0.0
Invalidates and refreshes all the cached data and metadata of the given table or view.
Invalidates and refreshes all the cached data and metadata of the given table or view. For Hive metastore table, the metadata is refreshed. For data source tables, the schema will not be inferred and refreshed.
If this table is cached as an InMemoryRelation, drop the original cached version and make the new version cached lazily.
is either a qualified or unqualified name that designates a table/view. If no database identifier is provided, it refers to a temporary view or a table/view in the current database.
2.0.0
Sets the current default database in this session.
Sets the current default database in this session.
Checks if the table or view with the specified name exists in the specified database.
Checks if the table or view with the specified name exists in the specified database.
is a name that designates a database.
is an unqualified name that designates a table.
Checks if the table or view with the specified name exists.
Checks if the table or view with the specified name exists. This can either be a temporary view or a table/view.
is either a qualified or unqualified name that designates a table/view. If no database identifier is provided, it refers to a table/view in the current database.
Removes the specified table or view from the in-memory cache.
Removes the specified table or view from the in-memory cache.
is either a qualified or unqualified name that designates a table/view. If no database identifier is provided, it refers to a temporary view or a table/view in the current database.
2.0.0
(Scala-specific) Create a table from the given path based on a data source, a schema and a set of options.
(Scala-specific) Create a table from the given path based on a data source, a schema and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
(Since version 2.2.0) use createTable instead.
2.0.0
:: Experimental :: Create a table from the given path based on a data source, a schema and a set of options.
:: Experimental :: Create a table from the given path based on a data source, a schema and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
(Since version 2.2.0) use createTable instead.
2.0.0
(Scala-specific) Creates a table from the given path based on a data source and a set of options.
(Scala-specific) Creates a table from the given path based on a data source and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
(Since version 2.2.0) use createTable instead.
2.0.0
Creates a table from the given path based on a data source and a set of options.
Creates a table from the given path based on a data source and a set of options. Then, returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
(Since version 2.2.0) use createTable instead.
2.0.0
Creates a table from the given path based on a data source and returns the corresponding DataFrame.
Creates a table from the given path based on a data source and returns the corresponding DataFrame.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
(Since version 2.2.0) use createTable instead.
2.0.0
Creates a table from the given path and returns the corresponding DataFrame.
Creates a table from the given path and returns the corresponding DataFrame. It will use the default data source configured by spark.sql.sources.default.
is either a qualified or unqualified name that designates a table. If no database identifier is provided, it refers to a table in the current database.
(Since version 2.2.0) use createTable instead.
2.0.0
Internal implementation of the user-facing
Catalog
.