conf

Type Members

1. case class SparkEsMapperConf(esMappingId: Option[String], esMappingParent: Option[String], esMappingVersion: Option[String], esMappingVersionType: Option[String], esMappingRouting: Option[String], esMappingTTLInMillis: Option[String], esMappingTimestamp: Option[String]) extends Serializable with Product

   

   https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-mapping

   https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#cfg-mapping

   esMappingId

   The document field/property name containing the document id.

   esMappingParent

   The document field/property name containing the document parent. To specify a constant, use the <CONSTANT> format.

   esMappingVersion

   The document field/property name containing the document version. To specify a constant, use the <CONSTANT> format.

   esMappingVersionType

   Indicates the type of versioning used. http://www.elastic.co/guide/en/elasticsearch/reference/2.0/docs-index_.html#_version_types If es.mapping.version is undefined (default), its value is unspecified. If es.mapping.version is specified, its value becomes external.

   esMappingRouting

   The document field/property name containing the document routing. To specify a constant, use the <CONSTANT> format.

   esMappingTTLInMillis

   The document field/property name containing the document time-to-live. To specify a constant, use the <CONSTANT> format.

   esMappingTimestamp

   The document field/property name containing the document timestamp. To specify a constant, use the <CONSTANT> format.

2. case class SparkEsTransportClientConf(transportAddresses: Seq[String], transportPort: Int, transportSettings: Map[String, String]) extends Serializable with Product

   

   Configurations for EsNativeDataFrameWriter's TransportClient.

   Configurations for EsNativeDataFrameWriter's TransportClient.

   transportAddresses

   The minimum set of hosts to connect to when establishing a client. CONFIG_CLIENT_TRANSPORT_SNIFF is enabled by default.

   transportPort

   The port to connect when establishing a client.

   transportSettings

   Miscellaneous settings for the TransportClient. Empty by default.

3. case class SparkEsWriteConf(bulkActions: Int, bulkSizeInMB: Int, concurrentRequests: Int, flushTimeoutInSeconds: Long) extends Serializable with Product

   

   Configurations for EsNativeDataFrameWriter's BulkProcessor.

   Configurations for EsNativeDataFrameWriter's BulkProcessor.

   bulkActions

   The number of IndexRequests to batch in one request.

   bulkSizeInMB

   The maximum size in MB of a batch.

   concurrentRequests

   The number of concurrent requests in flight.

   flushTimeoutInSeconds

   The maximum time in seconds to wait while closing a BulkProcessor.