Class

org.apache.spark.sql.pulsar

PulsarStreamDataWriter

Related Doc: package pulsar

Permalink

class PulsarStreamDataWriter extends PulsarRowWriter with DataWriter[InternalRow]

A DataWriter for Pulsar writing. One data writer will be created in each partition to process incoming rows.

Linear Supertypes
DataWriter[InternalRow], PulsarRowWriter, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. PulsarStreamDataWriter
  2. DataWriter
  3. PulsarRowWriter
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new PulsarStreamDataWriter(inputSchema: Seq[Attribute], clientConf: Map[String, AnyRef], producerConf: Map[String, AnyRef], topic: Option[String], adminUrl: String)

    Permalink

    inputSchema

    The attributes in the input data.

    clientConf

    Parameters to use for the Pulsar client.

    producerConf

    Parameters to use for the Pulsar producer.

    topic

    The topic that this data writer is targeting.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. def abort(): Unit

    Permalink
    Definition Classes
    PulsarStreamDataWriter → DataWriter

  5. lazy val admin: PulsarAdmin

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  6. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  7. def checkForErrors(): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  9. def close(): Unit

    Permalink

  10. def commit(): WriterCommitMessage

    Permalink
    Definition Classes
    PulsarStreamDataWriter → DataWriter

  11. lazy val dataType: DataType

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  12. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  13. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  14. var failedWrite: Throwable

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  15. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  16. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  17. def getProducer[T](tp: String): Producer[T]

    Permalink
    Definition Classes
    PulsarRowWriter

  18. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  19. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  20. val metaProj: UnsafeProjection

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  21. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  22. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  23. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  24. def producerClose(): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  25. def producerFlush(): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  26. lazy val pulsarSchema: Schema[_]

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  27. def sendRow(row: InternalRow): Unit

    Permalink

    Send the specified row to the producer, with a callback that will save any exception to failedWrite.

    Send the specified row to the producer, with a callback that will save any exception to failedWrite. Note that send is asynchronous; subclasses must flush() their producer before assuming the row is in Pulsar.

    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  28. lazy val serializer: PulsarSerializer

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  29. lazy val singleProducer: Producer[_]

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  30. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  31. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  32. val topic2Producer: Map[String, Producer[_]]

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  33. val valIsStruct: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  34. val valueProj: UnsafeProjection

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  35. val valueSchema: Seq[Attribute]

    Permalink
    Attributes
    protected
    Definition Classes
    PulsarRowWriter

  36. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  37. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  38. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  39. def write(row: InternalRow): Unit

    Permalink
    Definition Classes
    PulsarStreamDataWriter → DataWriter

Inherited from DataWriter[InternalRow]

Inherited from PulsarRowWriter

Inherited from AnyRef

Inherited from Any

Ungrouped