Package com.exasol.spark.s3

Class ExasolS3ScanBuilder

java.lang.Object

com.exasol.spark.s3.ExasolS3ScanBuilder

All Implemented Interfaces:

org.apache.spark.sql.connector.read.ScanBuilder, org.apache.spark.sql.connector.read.SupportsPushDownFilters, org.apache.spark.sql.connector.read.SupportsPushDownRequiredColumns

public class ExasolS3ScanBuilder
extends Object
implements org.apache.spark.sql.connector.read.ScanBuilder, org.apache.spark.sql.connector.read.SupportsPushDownFilters, org.apache.spark.sql.connector.read.SupportsPushDownRequiredColumns

A class that implements ScanBuilder interface for accessing S3 intermediate storage.

Constructor Summary

Constructors

Constructor	Description
ExasolS3ScanBuilder(com.exasol.spark.common.ExasolOptions options, org.apache.spark.sql.types.StructType schema, org.apache.spark.sql.util.CaseInsensitiveStringMap properties)	Creates a new instance of ExasolS3ScanBuilder.

Method Summary

Modifier and Type	Method	Description
org.apache.spark.sql.connector.read.Scan	build()
protected String	getScanQuery()	Returns SQL query that would be run on the Exasol database.
void	pruneColumns(org.apache.spark.sql.types.StructType schema)
org.apache.spark.sql.sources.Filter[]	pushedFilters()
org.apache.spark.sql.sources.Filter[]	pushFilters(org.apache.spark.sql.sources.Filter[] filters)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExasolS3ScanBuilder

public ExasolS3ScanBuilder(com.exasol.spark.common.ExasolOptions options,
                           org.apache.spark.sql.types.StructType schema,
                           org.apache.spark.sql.util.CaseInsensitiveStringMap properties)

Creates a new instance of ExasolS3ScanBuilder.

Parameters:

options - user provided options

schema - user-provided StructType schema

properties - original key-value properties map that is passed to delegating classes

Method Detail

pushFilters

public org.apache.spark.sql.sources.Filter[] pushFilters(org.apache.spark.sql.sources.Filter[] filters)

Specified by:

pushFilters in interface org.apache.spark.sql.connector.read.SupportsPushDownFilters

pushedFilters

public org.apache.spark.sql.sources.Filter[] pushedFilters()

Specified by:

pushedFilters in interface org.apache.spark.sql.connector.read.SupportsPushDownFilters

pruneColumns

public void pruneColumns(org.apache.spark.sql.types.StructType schema)

Specified by:

pruneColumns in interface org.apache.spark.sql.connector.read.SupportsPushDownRequiredColumns

build

public org.apache.spark.sql.connector.read.Scan build()

Specified by:

build in interface org.apache.spark.sql.connector.read.ScanBuilder

getScanQuery

protected String getScanQuery()

Returns SQL query that would be run on the Exasol database. This is enriched query that would add predicates or specific columns on top the user provided query or table. The result of this enriched query will be saved into the intermediate storage.

Returns:

Enriched SQL query for the intermediate storage.