The type of the MongoConfig
The type of the MongoConfig
Create a configuration from the values in the Map
, using the optional default configuration for any default values.
Create a configuration from the values in the Map
, using the optional default configuration for any default values.
Note: Values in the map do not need to be prefixed with the configPrefix.
a map of properties and their string values
the optional default configuration, used for determining the default values for the properties
the configuration
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
if true forces the writes to be inserts, even if a Dataset contains an _id
field. Default false
.
configures if the bulk operation is ordered property.
the data contains extended bson types and any datasets that contain structs that follow the extended bson
types will automatically be converted into native bson types. For example the following _id
field would be
converted into an ObjectId: {_id: {oid: "000000000000000000000000"}}
the write config
2.1.6
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
if true forces the writes to be inserts, even if a Dataset contains an _id
field. Default false
.
configures if the bulk operation is ordered property.
the write config
2.1.3
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
if true forces the writes to be inserts, even if a Dataset contains an _id
field. Default false
.
the write config
2.1.3
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
the write config
2.1.2
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
2.1
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
2.1
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
2.1
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the connection string used in the creation of this configuration
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
Create a configuration from the values in the Map
Create a configuration from the values in the Map
Note: Values in the map do not need to be prefixed with the configPrefix.
a map of properties and their string values
the configuration
Create a configuration from the sparkConf
Create a configuration from the sparkConf
Uses the prefixed properties that are set in the Spark configuration to create the config.
the spark configuration
overloaded parameters
the configuration
Create a configuration from the sparkConf
Create a configuration from the sparkConf
Uses the prefixed properties that are set in the Spark configuration to create the config.
the spark configuration
the configuration
Create a configuration from the sqlContext
Create a configuration from the sqlContext
Uses the prefixed properties that are set in the Spark configuration to create the config.
the SparkSession
the configuration
Create a configuration from the sparkContext
Create a configuration from the sparkContext
Uses the prefixed properties that are set in the Spark configuration to create the config.
the spark context
the configuration
The collection name property
The collection name property
The configuration prefix string for the current configuration scope
The configuration prefix string for the current configuration scope
Create a configuration easily from the Java API using the JavaSparkContext
Create a configuration easily from the Java API using the JavaSparkContext
Uses the prefixed properties that are set in the Spark configuration to create the config.
the SparkSession
the configuration
Create a configuration easily from the Java API using the JavaSparkContext
Create a configuration easily from the Java API using the JavaSparkContext
Uses the prefixed properties that are set in the Spark configuration to create the config.
the SQL context
the configuration
Create a configuration from the sparkConf
Create a configuration from the sparkConf
Uses the prefixed properties that are set in the Spark configuration to create the config.
the spark configuration
overloaded parameters
the configuration
Create a configuration easily from the Java API using the values in the Map
, using the optional default configuration for any
default values.
Create a configuration easily from the Java API using the values in the Map
, using the optional default configuration for any
default values.
Note: Values in the map do not need to be prefixed with the configPrefix.
a map of properties and their string values
the optional default configuration, used for determining the default values for the properties
the configuration
Create a configuration easily from the Java API using the values in the Map
Create a configuration easily from the Java API using the values in the Map
Note: Values in the map do not need to be prefixed with the configPrefix.
a map of properties and their string values
the configuration
Create a configuration easily from the Java API using the sparkConf
Create a configuration easily from the Java API using the sparkConf
Uses the prefixed properties that are set in the Spark configuration to create the config.
the spark configuration
the configuration
Create a configuration easily from the Java API using the JavaSparkContext
Create a configuration easily from the Java API using the JavaSparkContext
Uses the prefixed properties that are set in the Spark configuration to create the config.
the java spark context
the configuration
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
if true forces the writes to be inserts, even if a Dataset contains an _id
field. Default false
.
configures if the bulk operation is ordered property.
the data contains extended bson types and any datasets that contain structs that follow the extended bson
types will automatically be converted into native bson types. For example the following _id
field would be
converted into an ObjectId: {_id: {oid: "000000000000000000000000"}}
the write config
2.1.6
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
if true forces the writes to be inserts, even if a Dataset contains an _id
field. Default false
.
configures if the bulk operation is ordered property.
the write config
2.1.3
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
if true forces the writes to be inserts, even if a Dataset contains an _id
field. Default false
.
the write config
2.1.3
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
an optional shardKey in extended form: "{key: 1, key2: 1}"
. Used when upserting DataSets in sharded clusters.
the write config
2.1.2
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the maxBatchSize when performing a bulk update/insert. Defaults to 512.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
2.1
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
replaces the whole document, when saving a Dataset that contains an _id
field.
If false only updates / sets the fields declared in the Dataset.
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
2.1
Creates a WriteConfig
Creates a WriteConfig
the database name
the collection name
the optional connection string used in the creation of this configuration
the local threshold in milliseconds used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time plus the local threshold will be chosen.
the WriteConcern to use
the write config
The database name property
The database name property
The extended bson types property
The extended bson types property
The data contains extended bson types and any datasets that contain structs that follow the extended bson types will automatically be converted into native bson types.
Default: true
2.1.6
The force insert property
The force insert property
If true
, will always use inserts when saving data, even if a Dataset
contains an _id
field.
Default: None
2.1.3
Gets an options map from the SparkConf
Gets an options map from the SparkConf
the SparkConf
the options
The localThreshold property
The localThreshold property
The local threshold in milliseconds is used when choosing among multiple MongoDB servers to send a request. Only servers whose ping time is less than or equal to the server with the fastest ping time *plus* the local threshold will be chosen.
For example when choosing which MongoS to send a request through a localThreshold
of 0 would pick the MongoS with the fastest ping time.
Default: 15 ms
The maxBatchSize property
The maxBatchSize property
The maximum size of the batch when performing a bulk update / insert operation.
Default: 512
2.1
The mongo URI property
The mongo URI property
Represents a connection string.
Any values set in the connection string will override any default values for the configuration.
The ordered property
The ordered property
Configures the bulk operation ordered property.
Default: true
2.1.3
The replaceDocument property
The replaceDocument property
When saving a Dataset with an _id
field, the whole document will be replaced if true. Otherwise, only the fields in the Dataset will
be updated.
Default: true
2.1
The shardKey property
The shardKey property
The optional shardKey is required when upserting Datasets in sharded clusters
Default: None
2.1
Strip the prefix from options
Strip the prefix from options
options that may contain the prefix
prefixLess options
The WriteConcern
journal property
The WriteConcern
journal property
The WriteConcern
w property
The WriteConcern
w property
The WriteConcern
wTimeoutMS property
The WriteConcern
wTimeoutMS property
Create a configuration from the sqlContext
Create a configuration from the sqlContext
Uses the prefixed properties that are set in the Spark configuration to create the config.
the SQL context
the configuration
(Since version 2.0.0) As of Spark 2.0 SQLContext was replaced by SparkSession. Use the SparkSession method instead
The
WriteConfig
companion object1.0