Builder class that coordinates construction of a new SessionState.
Builder class that coordinates construction of a new SessionState.
The builder explicitly defines all components needed by the session state, and creates a session
state when build is called. Components should only be initialized once. This is not a problem
for most components as they are only used in the build function. However some components
(conf, catalog, functionRegistry, experimentalMethods & sqlParser) are as dependencies
for other components and are shared as a result. These components are defined as lazy vals to
make sure the component is created only once.
A developer can modify the builder by providing custom versions of components, or by using the hooks provided for the analyzer, optimizer & planner. There are some dependencies between the components (they are documented per dependency), a developer should respect these when making modifications in order to prevent initialization problems.
A parent SessionState can be used to initialize the new SessionState. The new session
state will clone the parent sessions state's conf, functionRegistry, experimentalMethods
and catalog fields. Note that the state is cloned when build is called, and not before.
Internal implementation of the user-facing Catalog.
Session shared FunctionResourceLoader.
Session shared FunctionResourceLoader.
Concrete implementation of a SessionStateBuilder.
Concrete implementation of a SessionStateBuilder.
A helper class that enables substitution using syntax like
${var}, ${system:var} and ${env:var}.
A helper class that enables substitution using syntax like
${var}, ${system:var} and ${env:var}.
Variable substitution is controlled by SQLConf.variableSubstituteEnabled.
All classes in this package are considered an internal API to Spark and are subject to change between minor releases.