Class/Object

com.mongodb.spark.config

ReadConfig

Related Docs: object ReadConfig | package config

Permalink

case class ReadConfig(databaseName: String, collectionName: String, connectionString: Option[String] = None, sampleSize: Int = ReadConfig.DefaultSampleSize, partitioner: MongoPartitioner = ReadConfig.DefaultPartitioner, partitionerOptions: Map[String, String] = ..., localThreshold: Int = ..., readPreferenceConfig: ReadPreferenceConfig = ReadPreferenceConfig(), readConcernConfig: ReadConcernConfig = ReadConcernConfig(), aggregationConfig: AggregationConfig = AggregationConfig(), registerSQLHelperFunctions: Boolean = ..., inferSchemaMapTypesEnabled: Boolean = ..., inferSchemaMapTypesMinimumKeys: Int = ..., pipelineIncludeNullFilters: Boolean = ..., pipelineIncludeFiltersAndProjections: Boolean = ..., samplePoolSize: Int = ReadConfig.DefaultSamplePoolSize) extends MongoCollectionConfig with MongoClassConfig with Product with Serializable

Read Configuration used when reading data from MongoDB

databaseName

the database name

collectionName

the collection name

connectionString

the optional connection string used in the creation of this configuration

sampleSize

a positive integer sample size to draw from the collection when inferring the schema

partitioner

the class name of the partitioner to use to create partitions

partitionerOptions

the configuration options for the partitioner

localThreshold

the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.

readPreferenceConfig

the readPreference configuration

readConcernConfig

the readConcern configuration

aggregationConfig

the aggregation configuration

registerSQLHelperFunctions

true to register sql helper functions

inferSchemaMapTypesEnabled

true to detect MapTypes when inferring Schema.

inferSchemaMapTypesMinimumKeys

the minimum number of keys before a document can be inferred as a MapType.

pipelineIncludeNullFilters

true to include and push down null and exists filters into the pipeline when using sql.

pipelineIncludeFiltersAndProjections

true to push down filters and projections into the pipeline when using sql.

samplePoolSize

the size of the pool to take a sample from, used when there is no $sample support or if there is a pushed down aggregation

Since

1.0

Linear Supertypes
Product, Equals, MongoClassConfig, MongoCollectionConfig, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ReadConfig
  2. Product
  3. Equals
  4. MongoClassConfig
  5. MongoCollectionConfig
  6. Serializable
  7. Serializable
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ReadConfig(databaseName: String, collectionName: String, connectionString: Option[String] = None, sampleSize: Int = ReadConfig.DefaultSampleSize, partitioner: MongoPartitioner = ReadConfig.DefaultPartitioner, partitionerOptions: Map[String, String] = ..., localThreshold: Int = ..., readPreferenceConfig: ReadPreferenceConfig = ReadPreferenceConfig(), readConcernConfig: ReadConcernConfig = ReadConcernConfig(), aggregationConfig: AggregationConfig = AggregationConfig(), registerSQLHelperFunctions: Boolean = ..., inferSchemaMapTypesEnabled: Boolean = ..., inferSchemaMapTypesMinimumKeys: Int = ..., pipelineIncludeNullFilters: Boolean = ..., pipelineIncludeFiltersAndProjections: Boolean = ..., samplePoolSize: Int = ReadConfig.DefaultSamplePoolSize)

    Permalink

    databaseName

    the database name

    collectionName

    the collection name

    connectionString

    the optional connection string used in the creation of this configuration

    sampleSize

    a positive integer sample size to draw from the collection when inferring the schema

    partitioner

    the class name of the partitioner to use to create partitions

    partitionerOptions

    the configuration options for the partitioner

    localThreshold

    the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.

    readPreferenceConfig

    the readPreference configuration

    readConcernConfig

    the readConcern configuration

    aggregationConfig

    the aggregation configuration

    registerSQLHelperFunctions

    true to register sql helper functions

    inferSchemaMapTypesEnabled

    true to detect MapTypes when inferring Schema.

    inferSchemaMapTypesMinimumKeys

    the minimum number of keys before a document can be inferred as a MapType.

    pipelineIncludeNullFilters

    true to include and push down null and exists filters into the pipeline when using sql.

    pipelineIncludeFiltersAndProjections

    true to push down filters and projections into the pipeline when using sql.

    samplePoolSize

    the size of the pool to take a sample from, used when there is no $sample support or if there is a pushed down aggregation

Type Members

  1. type Self = ReadConfig

    Permalink

    Defines Self as a type that can be used to return a copy of the object i.e.

    Defines Self as a type that can be used to return a copy of the object i.e. a different instance of the same type

    Definition Classes
    ReadConfigMongoClassConfig

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. val aggregationConfig: AggregationConfig

    Permalink

    the aggregation configuration

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. def asJavaOptions: Map[String, String]

    Permalink

    Creates a map of options representing the configuration

    Creates a map of options representing the configuration

    returns

    the map representing the configuration values

    Definition Classes
    ReadConfigMongoClassConfig
  7. def asOptions: Map[String, String]

    Permalink

    Creates a map of options representing this config.

    Creates a map of options representing this config.

    returns

    the map representing the config

    Definition Classes
    ReadConfigMongoClassConfig
  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. val collectionName: String

    Permalink

    the collection name

    the collection name

    Definition Classes
    ReadConfigMongoCollectionConfig
  10. val connectionString: Option[String]

    Permalink

    the optional connection string used in the creation of this configuration

  11. val databaseName: String

    Permalink

    the database name

    the database name

    Definition Classes
    ReadConfigMongoCollectionConfig
  12. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  15. val inferSchemaMapTypesEnabled: Boolean

    Permalink

    true to detect MapTypes when inferring Schema.

  16. val inferSchemaMapTypesMinimumKeys: Int

    Permalink

    the minimum number of keys before a document can be inferred as a MapType.

  17. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  18. val localThreshold: Int

    Permalink

    the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request.

    the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.

  19. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  20. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  21. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  22. val partitioner: MongoPartitioner

    Permalink

    the class name of the partitioner to use to create partitions

  23. val partitionerOptions: Map[String, String]

    Permalink

    the configuration options for the partitioner

  24. def pipeline: List[BsonDocument]

    Permalink

    returns

    the aggregation pipeline to use from the aggregation config or an empty list

  25. val pipelineIncludeFiltersAndProjections: Boolean

    Permalink

    true to push down filters and projections into the pipeline when using sql.

  26. val pipelineIncludeNullFilters: Boolean

    Permalink

    true to include and push down null and exists filters into the pipeline when using sql.

  27. def readConcern: ReadConcern

    Permalink

    The ReadConcern to use

    The ReadConcern to use

    returns

    the ReadConcern

  28. val readConcernConfig: ReadConcernConfig

    Permalink

    the readConcern configuration

  29. def readPreference: ReadPreference

    Permalink

    The ReadPreference to use

    The ReadPreference to use

    returns

    the ReadPreference

  30. val readPreferenceConfig: ReadPreferenceConfig

    Permalink

    the readPreference configuration

  31. val registerSQLHelperFunctions: Boolean

    Permalink

    true to register sql helper functions

  32. val samplePoolSize: Int

    Permalink

    the size of the pool to take a sample from, used when there is no $sample support or if there is a pushed down aggregation

  33. val sampleSize: Int

    Permalink

    a positive integer sample size to draw from the collection when inferring the schema

  34. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  35. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  36. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  37. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  38. def withOption(key: String, value: String): ReadConfig

    Permalink

    Creates a new config with the options applied

    Creates a new config with the options applied

    key

    the configuration key

    value

    the configuration value

    returns

    an updated config

    Definition Classes
    ReadConfigMongoClassConfig
  39. def withOptions(options: Map[String, String]): ReadConfig

    Permalink

    Creates a new config with the options applied

    Creates a new config with the options applied

    options

    a map of options to be applied to the config

    returns

    an updated config

    Definition Classes
    ReadConfigMongoClassConfig
  40. def withOptions(options: Map[String, String]): ReadConfig

    Permalink

    Creates a new config with the options applied

    Creates a new config with the options applied

    options

    a map of options to be applied to the config

    returns

    an updated config

    Definition Classes
    ReadConfigMongoClassConfig
  41. def withPipeline[B <: Bson](pipeline: Seq[B]): ReadConfig

    Permalink

    Returns a copy with the specified aggregation pipeline applied to the aggregation config

    Returns a copy with the specified aggregation pipeline applied to the aggregation config

    pipeline

    the aggregation pipeline to use

    returns

    the updated Read Config

Inherited from Product

Inherited from Equals

Inherited from MongoClassConfig

Inherited from MongoCollectionConfig

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped