Builder class that coordinates construction of a new SessionState.
Builder class that coordinates construction of a new SessionState.
The builder explicitly defines all components needed by the session state, and creates a session
state when build
is called. Components should only be initialized once. This is not a problem
for most components as they are only used in the build
function. However some components
(conf
, catalog
, functionRegistry
, experimentalMethods
& sqlParser
) are as dependencies
for other components and are shared as a result. These components are defined as lazy vals to
make sure the component is created only once.
A developer can modify the builder by providing custom versions of components, or by using the hooks provided for the analyzer, optimizer & planner. There are some dependencies between the components (they are documented per dependency), a developer should respect these when making modifications in order to prevent initialization problems.
A parent SessionState can be used to initialize the new SessionState. The new session
state will clone the parent sessions state's conf
, functionRegistry
, experimentalMethods
and catalog
fields. Note that the state is cloned when build
is called, and not before.
Internal implementation of the user-facing Catalog
.
Session shared FunctionResourceLoader.
Session shared FunctionResourceLoader.
Concrete implementation of a SessionStateBuilder.
Concrete implementation of a SessionStateBuilder.
A helper class that enables substitution using syntax like
${var}
, ${system:var}
and ${env:var}
.
A helper class that enables substitution using syntax like
${var}
, ${system:var}
and ${env:var}
.
Variable substitution is controlled by SQLConf.variableSubstituteEnabled
.
All classes in this package are considered an internal API to Spark and are subject to change between minor releases.