Package org.springframework.kafka.core

Class DefaultKafkaProducerFactory<K,V>

java.lang.Object
org.springframework.kafka.core.KafkaResourceFactory
org.springframework.kafka.core.DefaultKafkaProducerFactory<K,V>
Type Parameters:
K - the key type.
V - the value type.
All Implemented Interfaces:
EventListener, org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanNameAware, org.springframework.beans.factory.DisposableBean, org.springframework.context.ApplicationContextAware, org.springframework.context.ApplicationListener<org.springframework.context.event.ContextStoppedEvent>, ProducerFactory<K,V>
public class DefaultKafkaProducerFactory<K,V> extends KafkaResourceFactory implements ProducerFactory<K,V>, org.springframework.context.ApplicationContextAware, org.springframework.beans.factory.BeanNameAware, org.springframework.context.ApplicationListener<org.springframework.context.event.ContextStoppedEvent>, org.springframework.beans.factory.DisposableBean
The ProducerFactory implementation for a singleton shared Producer instance.

This implementation will return the same Producer instance (if transactions are not enabled) for the provided Map configs and optional Serializer implementations on each createProducer() invocation.

If you are using Serializers that have no-arg constructors and require no setup, then simplest to specify Serializer classes against ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG and ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG keys in the configs passed to the DefaultKafkaProducerFactory constructor.

If that is not possible, but you are sure that at least one of the following is true:

  • only one Producer will use the Serializers
  • you are using Serializers that may be shared between Producer instances (and specifically that their close() method is a no-op)
  • you are certain that there is no risk of any single Producer being closed while other Producer instances with the same Serializers are in use
then you can pass in Serializer instances for one or both of the key and value serializers.

If none of the above is true then you may provide a Supplier function for one or both Serializers which will be used to obtain Serializer(s) each time a Producer is created by the factory.

The Producer is wrapped and the underlying KafkaProducer instance is not actually closed when Producer.close() is invoked. The KafkaProducer is physically closed when DisposableBean.destroy() is invoked or when the application context publishes a ContextStoppedEvent. You can also invoke reset().

Setting setTransactionIdPrefix(String) enables transactions; in which case, a cache of producers is maintained; closing a producer returns it to the cache. The producers are closed and the cache is cleared when the factory is destroyed, the application context stopped, or the reset() method is called.

  • Constructor Details

    • DefaultKafkaProducerFactory

      public DefaultKafkaProducerFactory(Map<String,Object> configs)
      Construct a factory with the provided configuration.
      Parameters:
      configs - the configuration.

    • DefaultKafkaProducerFactory

      public DefaultKafkaProducerFactory(Map<String,Object> configs, @Nullable org.apache.kafka.common.serialization.Serializer<K> keySerializer, @Nullable org.apache.kafka.common.serialization.Serializer<V> valueSerializer)
      Construct a factory with the provided configuration and Serializers. Also configures a transactionIdPrefix as a value from the ProducerConfig.TRANSACTIONAL_ID_CONFIG if provided. This config is going to be overridden with a suffix for target Producer instance. The serializers' configure() methods will be called with the configuration map.
      Parameters:
      configs - the configuration.
      keySerializer - the key Serializer.
      valueSerializer - the value Serializer.

    • DefaultKafkaProducerFactory

      public DefaultKafkaProducerFactory(Map<String,Object> configs, @Nullable org.apache.kafka.common.serialization.Serializer<K> keySerializer, @Nullable org.apache.kafka.common.serialization.Serializer<V> valueSerializer, boolean configureSerializers)
      Construct a factory with the provided configuration and Serializers. Also configures a transactionIdPrefix as a value from the ProducerConfig.TRANSACTIONAL_ID_CONFIG if provided. This config is going to be overridden with a suffix for target Producer instance. The serializers' configure() methods will be called with the configuration map unless configureSerializers is false..
      Parameters:
      configs - the configuration.
      keySerializer - the key Serializer.
      valueSerializer - the value Serializer.
      configureSerializers - set to false if serializers are already fully configured.
      Since:
      2.8.7

    • DefaultKafkaProducerFactory

      public DefaultKafkaProducerFactory(Map<String,Object> configs, @Nullable Supplier<org.apache.kafka.common.serialization.Serializer<K>> keySerializerSupplier, @Nullable Supplier<org.apache.kafka.common.serialization.Serializer<V>> valueSerializerSupplier)
      Construct a factory with the provided configuration and Serializer Suppliers. Also configures a transactionIdPrefix as a value from the ProducerConfig.TRANSACTIONAL_ID_CONFIG if provided. This config is going to be overridden with a suffix for target Producer instance. When the suppliers are invoked to get an instance, the serializers' configure() methods will be called with the configuration map.
      Parameters:
      configs - the configuration.
      keySerializerSupplier - the key Serializer supplier function.
      valueSerializerSupplier - the value Serializer supplier function.
      Since:
      2.3

    • DefaultKafkaProducerFactory

      public DefaultKafkaProducerFactory(Map<String,Object> configs, @Nullable Supplier<org.apache.kafka.common.serialization.Serializer<K>> keySerializerSupplier, @Nullable Supplier<org.apache.kafka.common.serialization.Serializer<V>> valueSerializerSupplier, boolean configureSerializers)
      Construct a factory with the provided configuration and Serializer Suppliers. Also configures a transactionIdPrefix as a value from the ProducerConfig.TRANSACTIONAL_ID_CONFIG if provided. This config is going to be overridden with a suffix for target Producer instance. When the suppliers are invoked to get an instance, the serializers' configure() methods will be called with the configuration map unless configureSerializers is false.
      Parameters:
      configs - the configuration.
      keySerializerSupplier - the key Serializer supplier function.
      valueSerializerSupplier - the value Serializer supplier function.
      configureSerializers - set to false if serializers are already fully configured.
      Since:
      2.8.7

  • Method Details

    • setApplicationContext

      public void setApplicationContext(org.springframework.context.ApplicationContext applicationContext) throws org.springframework.beans.BeansException
      Specified by:
      setApplicationContext in interface org.springframework.context.ApplicationContextAware
      Throws:
      org.springframework.beans.BeansException

    • setBeanName

      public void setBeanName(String name)
      Specified by:
      setBeanName in interface org.springframework.beans.factory.BeanNameAware

    • setKeySerializer

      public void setKeySerializer(@Nullable org.apache.kafka.common.serialization.Serializer<K> keySerializer)
      Set a key serializer. The serializer will be configured using the producer configuration, unless configureSerializers is false.
      Parameters:
      keySerializer - the key serializer.
      See Also:

    • setValueSerializer

      public void setValueSerializer(@Nullable org.apache.kafka.common.serialization.Serializer<V> valueSerializer)
      Set a value serializer. The serializer will be configured using the producer configuration, unless configureSerializers is false.
      Parameters:
      valueSerializer - the value serializer.
      See Also:

    • setKeySerializerSupplier

      public void setKeySerializerSupplier(Supplier<org.apache.kafka.common.serialization.Serializer<K>> keySerializerSupplier)
      Set a supplier to supply instances of the key serializer. The serializer will be configured using the producer configuration, unless configureSerializers is false.
      Parameters:
      keySerializerSupplier - the supplier.
      Since:
      2.8
      See Also:

    • setValueSerializerSupplier

      public void setValueSerializerSupplier(Supplier<org.apache.kafka.common.serialization.Serializer<V>> valueSerializerSupplier)
      Set a supplier to supply instances of the value serializer.
      Parameters:
      valueSerializerSupplier - the supplier. The serializer will be configured using the producer configuration, unless configureSerializers is false.
      Since:
      2.8
      See Also:

    • isConfigureSerializers

      public boolean isConfigureSerializers()
      If true (default), programmatically provided serializers (via constructor or setters) will be configured using the producer configuration. Set to false if the serializers are already fully configured.
      Returns:
      true to configure.
      Since:
      2.8.7
      See Also:

    • setConfigureSerializers

      public void setConfigureSerializers(boolean configureSerializers)
      Set to false (default true) to prevent programmatically provided serializers (via constructor or setters) from being configured using the producer configuration, e.g. if the serializers are already fully configured.
      Parameters:
      configureSerializers - false to not configure.
      Since:
      2.8.7
      See Also:

    • setPhysicalCloseTimeout

      public void setPhysicalCloseTimeout(int physicalCloseTimeout)
      The time to wait when physically closing the producer via the factory rather than closing the producer itself (when reset(), #closeProducerFor(String), or closeThreadBoundProducer() are invoked). Specified in seconds; default ProducerFactory.DEFAULT_PHYSICAL_CLOSE_TIMEOUT.
      Parameters:
      physicalCloseTimeout - the timeout in seconds.
      Since:
      1.0.7

    • getPhysicalCloseTimeout

      public Duration getPhysicalCloseTimeout()
      Get the physical close timeout.
      Specified by:
      getPhysicalCloseTimeout in interface ProducerFactory<K,V>
      Returns:
      the timeout.
      Since:
      2.5

    • setTransactionIdPrefix

      public final void setTransactionIdPrefix(String transactionIdPrefix)
      Set a prefix for the ProducerConfig.TRANSACTIONAL_ID_CONFIG config. By default a ProducerConfig.TRANSACTIONAL_ID_CONFIG value from configs is used as a prefix in the target producer configs.
      Parameters:
      transactionIdPrefix - the prefix.
      Since:
      1.3

    • getTransactionIdPrefix

      @Nullable public String getTransactionIdPrefix()
      Description copied from interface: ProducerFactory
      Return the transaction id prefix.
      Specified by:
      getTransactionIdPrefix in interface ProducerFactory<K,V>
      Returns:
      the prefix or null if not configured.

    • setProducerPerThread

      public void setProducerPerThread(boolean producerPerThread)
      Set to true to create a producer per thread instead of singleton that is shared by all clients. Clients must call closeThreadBoundProducer() to physically close the producer when it is no longer needed. These producers will not be closed by destroy() or reset().
      Parameters:
      producerPerThread - true for a producer per thread.
      Since:
      2.3
      See Also:

    • isProducerPerThread

      public boolean isProducerPerThread()
      Description copied from interface: ProducerFactory
      Return true when there is a producer per thread.
      Specified by:
      isProducerPerThread in interface ProducerFactory<K,V>
      Returns:
      the producer per thread.

    • setProducerPerConsumerPartition

      @Deprecated(since="3.0", forRemoval=true) public void setProducerPerConsumerPartition(boolean producerPerConsumerPartition)
      Deprecated, for removal: This API element is subject to removal in a future version.
      no longer necessary because org.springframework.kafka.listener.ContainerProperties.EOSMode#V1 is no longer supported.
      This is no longer needed now that only ContainerProperties.EOSMode.V2 is supported. Ignored.
      Parameters:
      producerPerConsumerPartition - false to revert.
      Since:
      1.3.7

    • isProducerPerConsumerPartition

      @Deprecated(since="3.0", forRemoval=true) public boolean isProducerPerConsumerPartition()
      Deprecated, for removal: This API element is subject to removal in a future version.
      no longer necessary because org.springframework.kafka.listener.ContainerProperties.EOSMode#V1 is no longer supported.
      This is no longer needed now that only ContainerProperties.EOSMode.V2 is supported. Ignored.
      Specified by:
      isProducerPerConsumerPartition in interface ProducerFactory<K,V>
      Returns:
      the producerPerConsumerPartition.
      Since:
      1.3.8

    • getKeySerializer

      @Nullable public org.apache.kafka.common.serialization.Serializer<K> getKeySerializer()
      Description copied from interface: ProducerFactory
      Return the configured key serializer (if provided as an object instead of a class name in the properties).
      Specified by:
      getKeySerializer in interface ProducerFactory<K,V>
      Returns:
      the serializer.

    • getValueSerializer

      @Nullable public org.apache.kafka.common.serialization.Serializer<V> getValueSerializer()
      Description copied from interface: ProducerFactory
      Return the configured value serializer (if provided as an object instead of a class name in the properties).
      Specified by:
      getValueSerializer in interface ProducerFactory<K,V>
      Returns:
      the serializer.

    • getKeySerializerSupplier

      public Supplier<org.apache.kafka.common.serialization.Serializer<K>> getKeySerializerSupplier()
      Description copied from interface: ProducerFactory
      Return a supplier for a key serializer. Useful for cloning to make a similar factory.
      Specified by:
      getKeySerializerSupplier in interface ProducerFactory<K,V>
      Returns:
      the supplier.

    • getValueSerializerSupplier

      public Supplier<org.apache.kafka.common.serialization.Serializer<V>> getValueSerializerSupplier()
      Description copied from interface: ProducerFactory
      Return a supplier for a value serializer. Useful for cloning to make a similar factory.
      Specified by:
      getValueSerializerSupplier in interface ProducerFactory<K,V>
      Returns:
      the supplier.

    • getConfigurationProperties

      public Map<String,Object> getConfigurationProperties()
      Return an unmodifiable reference to the configuration map for this factory. Useful for cloning to make a similar factory.
      Specified by:
      getConfigurationProperties in interface ProducerFactory<K,V>
      Returns:
      the configs.
      Since:
      1.3

    • getListeners

      public List<ProducerFactory.Listener<K,V>> getListeners()
      Get the current list of listeners.
      Specified by:
      getListeners in interface ProducerFactory<K,V>
      Returns:
      the listeners.
      Since:
      2.5

    • getPostProcessors

      public List<ProducerPostProcessor<K,V>> getPostProcessors()
      Description copied from interface: ProducerFactory
      Get the current list of post processors.
      Specified by:
      getPostProcessors in interface ProducerFactory<K,V>
      Returns:
      the post processors.

    • setMaxAge

      public void setMaxAge(Duration maxAge)
      Set the maximum age for a producer; useful when using transactions and the broker might expire a transactional.id due to inactivity.
      Parameters:
      maxAge - the maxAge to set
      Since:
      2.5.8

    • copyWithConfigurationOverride

      public ProducerFactory<K,V> copyWithConfigurationOverride(Map<String,Object> overrideProperties)
      Copy properties of the instance and the given properties to create a new producer factory.

      If the DefaultKafkaProducerFactory makes a copy of itself, the transaction id prefix is recovered from the properties. If you want to change the ID config, add a new ProducerConfig.TRANSACTIONAL_ID_CONFIG key to the override config.

      Specified by:
      copyWithConfigurationOverride in interface ProducerFactory<K,V>
      Parameters:
      overrideProperties - the properties to be applied to the new factory
      Returns:
      DefaultKafkaProducerFactory with properties applied
      See Also:

    • addListener

      public void addListener(ProducerFactory.Listener<K,V> listener)
      Add a listener.
      Specified by:
      addListener in interface ProducerFactory<K,V>
      Parameters:
      listener - the listener.
      Since:
      2.5

    • addListener

      public void addListener(int index, ProducerFactory.Listener<K,V> listener)
      Add a listener at a specific index.
      Specified by:
      addListener in interface ProducerFactory<K,V>
      Parameters:
      index - the index (list position).
      listener - the listener.
      Since:
      2.5

    • removeListener

      public boolean removeListener(ProducerFactory.Listener<K,V> listener)
      Remove a listener.
      Specified by:
      removeListener in interface ProducerFactory<K,V>
      Parameters:
      listener - the listener.
      Returns:
      true if removed.
      Since:
      2.5

    • addPostProcessor

      public void addPostProcessor(ProducerPostProcessor<K,V> postProcessor)
      Description copied from interface: ProducerFactory
      Add a post processor.
      Specified by:
      addPostProcessor in interface ProducerFactory<K,V>
      Parameters:
      postProcessor - the post processor.

    • removePostProcessor

      public boolean removePostProcessor(ProducerPostProcessor<K,V> postProcessor)
      Description copied from interface: ProducerFactory
      Remove a post processor.
      Specified by:
      removePostProcessor in interface ProducerFactory<K,V>
      Parameters:
      postProcessor - the post processor.
      Returns:
      true if removed.

    • updateConfigs

      public void updateConfigs(Map<String,Object> updates)
      Description copied from interface: ProducerFactory
      Update the producer configuration map; useful for situations such as credential rotation.
      Specified by:
      updateConfigs in interface ProducerFactory<K,V>
      Parameters:
      updates - the configuration properties to update.

    • removeConfig

      public void removeConfig(String configKey)
      Description copied from interface: ProducerFactory
      Remove the specified key from the configuration map.
      Specified by:
      removeConfig in interface ProducerFactory<K,V>
      Parameters:
      configKey - the key to remove.

    • transactionCapable

      public boolean transactionCapable()
      Description copied from interface: ProducerFactory
      Return true if the factory supports transactions.
      Specified by:
      transactionCapable in interface ProducerFactory<K,V>
      Returns:
      true if transactional.

    • destroy

      public void destroy()
      Specified by:
      destroy in interface org.springframework.beans.factory.DisposableBean

    • onApplicationEvent

      public void onApplicationEvent(org.springframework.context.event.ContextStoppedEvent event)
      Specified by:
      onApplicationEvent in interface org.springframework.context.ApplicationListener<K>

    • reset

      public void reset()
      Close the Producer(s) and clear the cache of transactional Producer(s).
      Specified by:
      reset in interface ProducerFactory<K,V>
      Since:
      2.2

    • createProducer

      public org.apache.kafka.clients.producer.Producer<K,V> createProducer()
      Description copied from interface: ProducerFactory
      Create a producer which will be transactional if the factory is so configured.
      Specified by:
      createProducer in interface ProducerFactory<K,V>
      Returns:
      the producer.
      See Also:

    • createProducer

      public org.apache.kafka.clients.producer.Producer<K,V> createProducer(@Nullable String txIdPrefixArg)
      Description copied from interface: ProducerFactory
      Create a producer with an overridden transaction id prefix.
      Specified by:
      createProducer in interface ProducerFactory<K,V>
      Parameters:
      txIdPrefixArg - the transaction id prefix.
      Returns:
      the producer.

    • createNonTransactionalProducer

      public org.apache.kafka.clients.producer.Producer<K,V> createNonTransactionalProducer()
      Description copied from interface: ProducerFactory
      Create a non-transactional producer.
      Specified by:
      createNonTransactionalProducer in interface ProducerFactory<K,V>
      Returns:
      the producer.
      See Also:

    • createKafkaProducer

      protected org.apache.kafka.clients.producer.Producer<K,V> createKafkaProducer()
      Subclasses must return a raw producer which will be wrapped in a DefaultKafkaProducerFactory.CloseSafeProducer.
      Returns:
      the producer.

    • removeProducer

      protected final boolean removeProducer(DefaultKafkaProducerFactory.CloseSafeProducer<K,V> producerToRemove, Duration timeout)
      Remove the single shared producer and a thread-bound instance if present.
      Parameters:
      producerToRemove - the producer.
      timeout - the close timeout.
      Returns:
      always true.
      Since:
      2.2.13

    • createTransactionalProducer

      protected org.apache.kafka.clients.producer.Producer<K,V> createTransactionalProducer()
      Subclasses must return a producer from the getCache() or a new raw producer wrapped in a DefaultKafkaProducerFactory.CloseSafeProducer.
      Returns:
      the producer - cannot be null.
      Since:
      1.3

    • createTransactionalProducer

      protected org.apache.kafka.clients.producer.Producer<K,V> createTransactionalProducer(String txIdPrefix)

    • createRawProducer

      protected org.apache.kafka.clients.producer.Producer<K,V> createRawProducer(Map<String,Object> rawConfigs)

    • getCache

      @Nullable protected BlockingQueue<DefaultKafkaProducerFactory.CloseSafeProducer<K,V>> getCache()

    • getCache

      @Nullable protected BlockingQueue<DefaultKafkaProducerFactory.CloseSafeProducer<K,V>> getCache(String txIdPrefix)

    • closeThreadBoundProducer

      public void closeThreadBoundProducer()
      When using setProducerPerThread(boolean) (true), call this method to close and release this thread's producer. Thread bound producers are not closed by destroy() or reset() methods.
      Specified by:
      closeThreadBoundProducer in interface ProducerFactory<K,V>
      Since:
      2.3
      See Also:

    • getProducerConfigs

      protected Map<String,Object> getProducerConfigs()
      Return the configuration of a producer.
      Returns:
      the configuration of a producer.
      Since:
      2.8.3
      See Also:

    • getTxProducerConfigs

      protected Map<String,Object> getTxProducerConfigs(String transactionId)
      Return the configuration of a transactional producer.
      Parameters:
      transactionId - the transactionId.
      Returns:
      the configuration of a transactional producer.
      Since:
      2.8.3
      See Also:
      • doCreateTxProducer(String, String, BiPredicate)