public class KinesisProducerConfiguration extends Object
KinesisProducer
. See each each individual set
method for details about each parameter.Modifier and Type | Class and Description |
---|---|
static class |
KinesisProducerConfiguration.ThreadingModel
Configures the threading model used by the native process for handling requests to AWS Services.
|
Constructor and Description |
---|
KinesisProducerConfiguration() |
Modifier and Type | Method and Description |
---|---|
void |
addAdditionalMetricsDimension(String key,
String value,
String granularity)
Add an additional, custom dimension to the metrics emitted by the KPL.
|
protected com.amazonaws.services.kinesis.producer.protobuf.Config.Configuration.Builder |
additionalConfigsToProtobuf(com.amazonaws.services.kinesis.producer.protobuf.Config.Configuration.Builder builder) |
static KinesisProducerConfiguration |
fromProperties(Properties props)
Load configuration from a
Properties object. |
static KinesisProducerConfiguration |
fromPropertiesFile(String path)
Load configuration from a properties file.
|
long |
getAggregationMaxCount()
Maximum number of items to pack into an aggregated record.
|
long |
getAggregationMaxSize()
Maximum number of bytes to pack into an aggregated Kinesis record.
|
String |
getCloudwatchEndpoint()
Use a custom CloudWatch endpoint.
|
long |
getCloudwatchPort()
Server port to connect to for CloudWatch.
|
long |
getCollectionMaxCount()
Maximum number of items to pack into an PutRecords request.
|
long |
getCollectionMaxSize()
Maximum amount of data to send with a PutRecords request.
|
long |
getConnectTimeout()
Timeout (milliseconds) for establishing TLS connections.
|
com.amazonaws.auth.AWSCredentialsProvider |
getCredentialsProvider()
AWSCredentialsProvider that supplies credentials used to put records to Kinesis. |
long |
getCredentialsRefreshDelay()
How often to refresh credentials (in milliseconds).
|
String |
getKinesisEndpoint()
Use a custom Kinesis endpoint.
|
long |
getKinesisPort()
Server port to connect to for Kinesis.
|
String |
getLogLevel()
Minimum level of logs.
|
long |
getMaxConnections()
Maximum number of connections to open to the backend.
|
com.amazonaws.auth.AWSCredentialsProvider |
getMetricsCredentialsProvider()
AWSCredentialsProvider that supplies credentials used to upload
metrics to CloudWatch. |
String |
getMetricsGranularity()
Controls the granularity of metrics that are uploaded to CloudWatch.
|
String |
getMetricsLevel()
Controls the number of metrics that are uploaded to CloudWatch.
|
String |
getMetricsNamespace()
The namespace to upload metrics under.
|
long |
getMetricsUploadDelay()
Delay (in milliseconds) between each metrics upload.
|
long |
getMinConnections()
Minimum number of connections to keep open to the backend.
|
String |
getNativeExecutable()
Path to the native KPL binary.
|
long |
getRateLimit()
Limits the maximum allowed put rate for a shard, as a percentage of the backend limits.
|
long |
getRecordMaxBufferedTime()
Maximum amount of time (milliseconds) a record may spend being buffered before it gets
sent.
|
long |
getRecordTtl()
Set a time-to-live on records (milliseconds).
|
String |
getRegion()
Which region to send records to.
|
long |
getRequestTimeout()
The maximum total time (milliseconds) elapsed between when we begin a HTTP request and
receiving all of the response.
|
String |
getTempDirectory()
Temp directory into which to extract the native binaries.
|
KinesisProducerConfiguration.ThreadingModel |
getThreadingModel()
Returns the threading model that the native process will use to handle requests to AWS services
|
int |
getThreadPoolSize()
This configures the maximum number of threads the thread pool in the native process will use.
|
boolean |
isAggregationEnabled()
Enable aggregation.
|
boolean |
isEnableCoreDumps()
This has no effect on Windows.
|
boolean |
isFailIfThrottled()
If true, throttled puts are not retried.
|
boolean |
isVerifyCertificate()
Verify SSL certificates.
|
KinesisProducerConfiguration |
setAggregationEnabled(boolean val)
Enable aggregation.
|
KinesisProducerConfiguration |
setAggregationMaxCount(long val)
Maximum number of items to pack into an aggregated record.
|
KinesisProducerConfiguration |
setAggregationMaxSize(long val)
Maximum number of bytes to pack into an aggregated Kinesis record.
|
KinesisProducerConfiguration |
setCloudwatchEndpoint(String val)
Use a custom CloudWatch endpoint.
|
KinesisProducerConfiguration |
setCloudwatchPort(long val)
Server port to connect to for CloudWatch.
|
KinesisProducerConfiguration |
setCollectionMaxCount(long val)
Maximum number of items to pack into an PutRecords request.
|
KinesisProducerConfiguration |
setCollectionMaxSize(long val)
Maximum amount of data to send with a PutRecords request.
|
KinesisProducerConfiguration |
setConnectTimeout(long val)
Timeout (milliseconds) for establishing TLS connections.
|
KinesisProducerConfiguration |
setCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider credentialsProvider)
AWSCredentialsProvider that supplies credentials used to put records to Kinesis. |
KinesisProducerConfiguration |
setCredentialsRefreshDelay(long val)
How often to refresh credentials (in milliseconds).
|
KinesisProducerConfiguration |
setEnableCoreDumps(boolean val)
This has no effect on Windows.
|
KinesisProducerConfiguration |
setFailIfThrottled(boolean val)
If true, throttled puts are not retried.
|
KinesisProducerConfiguration |
setKinesisEndpoint(String val)
Use a custom Kinesis endpoint.
|
KinesisProducerConfiguration |
setKinesisPort(long val)
Server port to connect to for Kinesis.
|
KinesisProducerConfiguration |
setLogLevel(String val)
Minimum level of logs.
|
KinesisProducerConfiguration |
setMaxConnections(long val)
Maximum number of connections to open to the backend.
|
KinesisProducerConfiguration |
setMetricsCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider metricsCredentialsProvider)
AWSCredentialsProvider that supplies credentials used to upload
metrics to CloudWatch. |
KinesisProducerConfiguration |
setMetricsGranularity(String val)
Controls the granularity of metrics that are uploaded to CloudWatch.
|
KinesisProducerConfiguration |
setMetricsLevel(String val)
Controls the number of metrics that are uploaded to CloudWatch.
|
KinesisProducerConfiguration |
setMetricsNamespace(String val)
The namespace to upload metrics under.
|
KinesisProducerConfiguration |
setMetricsUploadDelay(long val)
Delay (in milliseconds) between each metrics upload.
|
KinesisProducerConfiguration |
setMinConnections(long val)
Minimum number of connections to keep open to the backend.
|
KinesisProducerConfiguration |
setNativeExecutable(String val)
Path to the native KPL binary.
|
KinesisProducerConfiguration |
setRateLimit(long val)
Limits the maximum allowed put rate for a shard, as a percentage of the backend limits.
|
KinesisProducerConfiguration |
setRecordMaxBufferedTime(long val)
Maximum amount of time (milliseconds) a record may spend being buffered before it gets
sent.
|
KinesisProducerConfiguration |
setRecordTtl(long val)
Set a time-to-live on records (milliseconds).
|
KinesisProducerConfiguration |
setRegion(String val)
Which region to send records to.
|
KinesisProducerConfiguration |
setRequestTimeout(long val)
The maximum total time (milliseconds) elapsed between when we begin a HTTP request and
receiving all of the response.
|
KinesisProducerConfiguration |
setTempDirectory(String val)
Temp directory into which to extract the native binaries.
|
KinesisProducerConfiguration |
setThreadingModel(KinesisProducerConfiguration.ThreadingModel threadingModel)
Sets the threading model that the native process will use.
|
KinesisProducerConfiguration |
setThreadingModel(String threadingModel)
Sets the threading model that the native process will use.
|
KinesisProducerConfiguration |
setThreadPoolSize(int threadPoolSize)
Sets the maximum number of threads that the native process' thread pool will be configured with.
|
KinesisProducerConfiguration |
setVerifyCertificate(boolean val)
Verify SSL certificates.
|
protected com.amazonaws.services.kinesis.producer.protobuf.Messages.Message |
toProtobufMessage() |
public void addAdditionalMetricsDimension(String key, String value, String granularity)
For example, you can make the KPL emit per-host metrics by adding HostName as the key and the domain name of the current host as the value.
The granularity of the custom dimension must be specified with the granularity parameter. The options are
"global", "stream" and "shard", just like setMetricsGranularity(String)
. If global is chosen, the custom
dimension will be inserted before the stream name; if stream is chosen then the custom metric will be inserted
after the stream name, but before the shard id. Lastly, if shard is chosen, the custom metric is inserted after
the shard id.
For example, if you want to see how different hosts are affecting a single stream, you can choose a granularity of stream for your HostName custom dimension. This will produce per-host metrics for every stream. On the other hand, if you want to see how a single host is distributing its load across different streams, you can choose a granularity of global. This will produce per-stream metrics for each host.
Note that custom dimensions will multiplicatively increase the number of metrics emitted by the KPL into CloudWatch.
key
- Name of the dimension, e.g. "HostName". Length must be between 1 and 255.value
- Value of the dimension, e.g. "my-host-1.my-domain.com". Length must be between 1 and 255.granularity
- Granularity of the custom dimension, must be one of "global", "stream" or "shard"IllegalArgumentException
- If granularity is not one of the allowed values.public com.amazonaws.auth.AWSCredentialsProvider getCredentialsProvider()
AWSCredentialsProvider
that supplies credentials used to put records to Kinesis. These credentials will
also be used to upload metrics to CloudWatch, unless setMetricsCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider)
is used to provide
separate credentials for that.public KinesisProducerConfiguration setCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider credentialsProvider)
AWSCredentialsProvider
that supplies credentials used to put records to Kinesis.
These credentials will also be used to upload metrics to CloudWatch, unless
setMetricsCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider)
is used to provide separate credentials for that.
Defaults to an instance of DefaultAWSCredentialsProviderChain
public com.amazonaws.auth.AWSCredentialsProvider getMetricsCredentialsProvider()
AWSCredentialsProvider
that supplies credentials used to upload
metrics to CloudWatch. If not given, the credentials used to put records
to Kinesis are also used for CloudWatch.public KinesisProducerConfiguration setMetricsCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider metricsCredentialsProvider)
AWSCredentialsProvider
that supplies credentials used to upload
metrics to CloudWatch.
If not given, the credentials used to put records to Kinesis are also used for CloudWatch.
public static KinesisProducerConfiguration fromPropertiesFile(String path)
The values loaded are checked against any constraints that each respective field may have. If there are invalid values an IllegalArgumentException will be thrown.
path
- Path to the properties file containing KPL config.KinesisProducerConfiguration
instance containing values
loaded from the specified file.IllegalArgumentException
- If one or more config values are invalid.public static KinesisProducerConfiguration fromProperties(Properties props)
Properties
object. Any fields not found
in the properties instance will take on default values.
The values loaded are checked against any constraints that each respective field may have. If there are invalid values an IllegalArgumentException will be thrown.
props
- Properties
object containing KPL config.KinesisProducerConfiguration
instance containing values
loaded from the specified file.IllegalArgumentException
- If one or more config values are invalid.protected com.amazonaws.services.kinesis.producer.protobuf.Config.Configuration.Builder additionalConfigsToProtobuf(com.amazonaws.services.kinesis.producer.protobuf.Config.Configuration.Builder builder)
public boolean isAggregationEnabled()
If your records are small, enabling aggregation will allow you to put many more records than you would otherwise be able to for a shard before getting throttled.
Default: true
public long getAggregationMaxCount()
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
Default: 4294967295
Minimum: 1
Maximum (inclusive): 9223372036854775807
public long getAggregationMaxSize()
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
If a record has more data by itself than this limit, it will bypass the aggregator. Note the backend enforces a limit of 50KB on record size. If you set this beyond 50KB, oversize records will be rejected at the backend.
Default: 51200
Minimum: 64
Maximum (inclusive): 1048576
public String getCloudwatchEndpoint()
Note this does not accept protocols or paths, only host names or ip addresses. There is no way to disable TLS. The KPL always connects with TLS.
Expected pattern: ^([A-Za-z0-9-\\.]+)?$
public long getCloudwatchPort()
Default: 443
Minimum: 1
Maximum (inclusive): 65535
public long getCollectionMaxCount()
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
Default: 500
Minimum: 1
Maximum (inclusive): 500
public long getCollectionMaxSize()
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
Records larger than the limit will still be sent, but will not be grouped with others.
Default: 5242880
Minimum: 52224
Maximum (inclusive): 9223372036854775807
public long getConnectTimeout()
Default: 6000
Minimum: 100
Maximum (inclusive): 300000
public long getCredentialsRefreshDelay()
During a refresh, credentials are retrieved from any SDK credentials providers attached to the wrapper and pushed to the core.
Default: 5000
Minimum: 1
Maximum (inclusive): 300000
public boolean isEnableCoreDumps()
If set to true, the KPL native process will attempt to raise its own core file size soft limit to 128MB, or the hard limit, whichever is lower. If the soft limit is already at or above the target amount, it is not changed.
Note that even if the limit is successfully raised (or already sufficient), it does not guarantee that core files will be written on a crash, since that is dependent on operation system settings that's beyond the control of individual processes.
Default: false
public boolean isFailIfThrottled()
If false, the KPL will automatically retry throttled puts. The KPL performs backoff for shards that it has received throttling errors from, and will avoid flooding them with retries. Note that records may fail from expiration (see record_ttl) if they get delayed for too long because of throttling.
Default: false
public String getKinesisEndpoint()
Note this does not accept protocols or paths, only host names or ip addresses. There is no way to disable TLS. The KPL always connects with TLS.
Expected pattern: ^([A-Za-z0-9-\\.]+)?$
public long getKinesisPort()
Default: 443
Minimum: 1
Maximum (inclusive): 65535
public String getLogLevel()
Default: info
Expected pattern: info|warning|error
public long getMaxConnections()
Setting this too high may impact latency and consume additional resources without increasing throughput.
Default: 24
Minimum: 1
Maximum (inclusive): 256
public String getMetricsGranularity()
When "shard" is selected, metrics are emitted with the stream name and shard id as dimensions. On top of this, the same metric is also emitted with only the stream name dimension, and lastly, without the stream name. This means for a particular metric, 2 streams with 2 shards (each) will produce 7 CloudWatch metrics, one for each shard, one for each stream, and one overall, all describing the same statistics, but at different levels of granularity.
When "stream" is selected, per shard metrics are not uploaded; when "global" is selected, only the total aggregate for all streams and all shards are uploaded.
Consider reducing the granularity if you're not interested in shard-level metrics, or if you have a large number of shards.
If you only have 1 stream, select "global"; the global data will be equivalent to that for the stream.
Refer to the metrics documentation for details about each metric.
Default: shard
Expected pattern: global|stream|shard
public String getMetricsLevel()
"none" disables all metrics.
"summary" enables the following metrics: UserRecordsPut, KinesisRecordsPut, ErrorsByCode, AllErrors, BufferingTime.
"detailed" enables all remaining metrics.
Refer to the metrics documentation for details about each metric.
Default: detailed
Expected pattern: none|summary|detailed
public String getMetricsNamespace()
If you have multiple applications running the KPL under the same AWS account, you should use a different namespace for each application.
If you are also using the KCL, you may wish to use the application name you have configured for the KCL as the the namespace here. This way both your KPL and KCL metrics show up under the same namespace.
Default: KinesisProducerLibrary
Expected pattern: (?!AWS/).{1,255}
public long getMetricsUploadDelay()
For testing only. There is no benefit in setting this lower or higher in production.
Default: 60000
Minimum: 1
Maximum (inclusive): 60000
public long getMinConnections()
There should be no need to increase this in general.
Default: 1
Minimum: 1
Maximum (inclusive): 16
public String getNativeExecutable()
public long getRateLimit()
The rate limit prevents the producer from sending data too fast to a shard. Such a limit is useful for reducing bandwidth and CPU cycle wastage from sending requests that we know are going to fail from throttling.
Kinesis enforces limits on both the number of records and number of bytes per second. This setting applies to both.
The default value of 150% is chosen to allow a single producer instance to completely saturate the allowance for a shard. This is an aggressive setting. If you prefer to reduce throttling errors rather than completely saturate the shard, consider reducing this setting.
Default: 150
Minimum: 1
Maximum (inclusive): 9223372036854775807
public long getRecordMaxBufferedTime()
This setting provides coarse ordering among records - any two records will be reordered by no more than twice this amount (assuming no failures and retries and equal network latency).
The library makes a best effort to enforce this time, but cannot guarantee that it will be precisely met. In general, if the CPU is not overloaded, the library will meet this deadline to within 10ms.
Failures and retries can additionally increase the amount of time records spend in the KPL. If your application cannot tolerate late records, use the record_ttl setting to drop records that do not get transmitted in time.
Setting this too low can negatively impact throughput.
Default: 100
Maximum (inclusive): 9223372036854775807
public long getRecordTtl()
This setting is useful if your application cannot or does not wish to tolerate late records. Records will still incur network latency after they leave the KPL, so take that into consideration when choosing a value for this setting.
If you do not wish to lose records and prefer to retry indefinitely, set record_ttl to a large value like INT_MAX. This has the potential to cause head-of-line blocking if network issues or throttling occur. You can respond to such situations by using the metrics reporting functions of the KPL. You may also set fail_if_throttled to true to prevent automatic retries in case of throttling.
Default: 30000
Minimum: 100
Maximum (inclusive): 9223372036854775807
public String getRegion()
If you do not specify the region and are running in EC2, the library will use the region the instance is in.
The region is also used to sign requests.
Expected pattern: ^([a-z]+-([a-z]+-)?[a-z]+-[0-9])?$
public long getRequestTimeout()
Note that a timed-out request may actually succeed at the backend. Retrying then leads to duplicates. Setting the timeout too low will therefore increase the probability of duplicates.
Default: 6000
Minimum: 100
Maximum (inclusive): 600000
public String getTempDirectory()
If not specified, defaults to /tmp in Unix. (Windows TBD)
public boolean isVerifyCertificate()
Default: true
public KinesisProducerConfiguration.ThreadingModel getThreadingModel()
KinesisProducerConfiguration.ThreadingModel
the native process will use.public int getThreadPoolSize()
getThreadingModel()
is set to KinesisProducerConfiguration.ThreadingModel.POOLED
.
public KinesisProducerConfiguration setAggregationEnabled(boolean val)
If your records are small, enabling aggregation will allow you to put many more records than you would otherwise be able to for a shard before getting throttled.
Default: true
public KinesisProducerConfiguration setAggregationMaxCount(long val)
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
Default: 4294967295
Minimum: 1
Maximum (inclusive): 9223372036854775807
public KinesisProducerConfiguration setAggregationMaxSize(long val)
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
If a record has more data by itself than this limit, it will bypass the aggregator. Note the backend enforces a limit of 50KB on record size. If you set this beyond 50KB, oversize records will be rejected at the backend.
Default: 51200
Minimum: 64
Maximum (inclusive): 1048576
public KinesisProducerConfiguration setCloudwatchEndpoint(String val)
Note this does not accept protocols or paths, only host names or ip addresses. There is no way to disable TLS. The KPL always connects with TLS.
Expected pattern: ^([A-Za-z0-9-\\.]+)?$
public KinesisProducerConfiguration setCloudwatchPort(long val)
Default: 443
Minimum: 1
Maximum (inclusive): 65535
public KinesisProducerConfiguration setCollectionMaxCount(long val)
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
Default: 500
Minimum: 1
Maximum (inclusive): 500
public KinesisProducerConfiguration setCollectionMaxSize(long val)
There should be normally no need to adjust this. If you want to limit the time records spend buffering, look into record_max_buffered_time instead.
Records larger than the limit will still be sent, but will not be grouped with others.
Default: 5242880
Minimum: 52224
Maximum (inclusive): 9223372036854775807
public KinesisProducerConfiguration setConnectTimeout(long val)
Default: 6000
Minimum: 100
Maximum (inclusive): 300000
public KinesisProducerConfiguration setCredentialsRefreshDelay(long val)
During a refresh, credentials are retrieved from any SDK credentials providers attached to the wrapper and pushed to the core.
Default: 5000
Minimum: 1
Maximum (inclusive): 300000
public KinesisProducerConfiguration setEnableCoreDumps(boolean val)
If set to true, the KPL native process will attempt to raise its own core file size soft limit to 128MB, or the hard limit, whichever is lower. If the soft limit is already at or above the target amount, it is not changed.
Note that even if the limit is successfully raised (or already sufficient), it does not guarantee that core files will be written on a crash, since that is dependent on operation system settings that's beyond the control of individual processes.
Default: false
public KinesisProducerConfiguration setFailIfThrottled(boolean val)
If false, the KPL will automatically retry throttled puts. The KPL performs backoff for shards that it has received throttling errors from, and will avoid flooding them with retries. Note that records may fail from expiration (see record_ttl) if they get delayed for too long because of throttling.
Default: false
public KinesisProducerConfiguration setKinesisEndpoint(String val)
Note this does not accept protocols or paths, only host names or ip addresses. There is no way to disable TLS. The KPL always connects with TLS.
Expected pattern: ^([A-Za-z0-9-\\.]+)?$
public KinesisProducerConfiguration setKinesisPort(long val)
Default: 443
Minimum: 1
Maximum (inclusive): 65535
public KinesisProducerConfiguration setLogLevel(String val)
Default: info
Expected pattern: info|warning|error
public KinesisProducerConfiguration setMaxConnections(long val)
Setting this too high may impact latency and consume additional resources without increasing throughput.
Default: 24
Minimum: 1
Maximum (inclusive): 256
public KinesisProducerConfiguration setMetricsGranularity(String val)
When "shard" is selected, metrics are emitted with the stream name and shard id as dimensions. On top of this, the same metric is also emitted with only the stream name dimension, and lastly, without the stream name. This means for a particular metric, 2 streams with 2 shards (each) will produce 7 CloudWatch metrics, one for each shard, one for each stream, and one overall, all describing the same statistics, but at different levels of granularity.
When "stream" is selected, per shard metrics are not uploaded; when "global" is selected, only the total aggregate for all streams and all shards are uploaded.
Consider reducing the granularity if you're not interested in shard-level metrics, or if you have a large number of shards.
If you only have 1 stream, select "global"; the global data will be equivalent to that for the stream.
Refer to the metrics documentation for details about each metric.
Default: shard
Expected pattern: global|stream|shard
public KinesisProducerConfiguration setMetricsLevel(String val)
"none" disables all metrics.
"summary" enables the following metrics: UserRecordsPut, KinesisRecordsPut, ErrorsByCode, AllErrors, BufferingTime.
"detailed" enables all remaining metrics.
Refer to the metrics documentation for details about each metric.
Default: detailed
Expected pattern: none|summary|detailed
public KinesisProducerConfiguration setMetricsNamespace(String val)
If you have multiple applications running the KPL under the same AWS account, you should use a different namespace for each application.
If you are also using the KCL, you may wish to use the application name you have configured for the KCL as the the namespace here. This way both your KPL and KCL metrics show up under the same namespace.
Default: KinesisProducerLibrary
Expected pattern: (?!AWS/).{1,255}
public KinesisProducerConfiguration setMetricsUploadDelay(long val)
For testing only. There is no benefit in setting this lower or higher in production.
Default: 60000
Minimum: 1
Maximum (inclusive): 60000
public KinesisProducerConfiguration setMinConnections(long val)
There should be no need to increase this in general.
Default: 1
Minimum: 1
Maximum (inclusive): 16
public KinesisProducerConfiguration setNativeExecutable(String val)
public KinesisProducerConfiguration setRateLimit(long val)
The rate limit prevents the producer from sending data too fast to a shard. Such a limit is useful for reducing bandwidth and CPU cycle wastage from sending requests that we know are going to fail from throttling.
Kinesis enforces limits on both the number of records and number of bytes per second. This setting applies to both.
The default value of 150% is chosen to allow a single producer instance to completely saturate the allowance for a shard. This is an aggressive setting. If you prefer to reduce throttling errors rather than completely saturate the shard, consider reducing this setting.
Default: 150
Minimum: 1
Maximum (inclusive): 9223372036854775807
public KinesisProducerConfiguration setRecordMaxBufferedTime(long val)
This setting provides coarse ordering among records - any two records will be reordered by no more than twice this amount (assuming no failures and retries and equal network latency).
The library makes a best effort to enforce this time, but cannot guarantee that it will be precisely met. In general, if the CPU is not overloaded, the library will meet this deadline to within 10ms.
Failures and retries can additionally increase the amount of time records spend in the KPL. If your application cannot tolerate late records, use the record_ttl setting to drop records that do not get transmitted in time.
Setting this too low can negatively impact throughput.
Default: 100
Maximum (inclusive): 9223372036854775807
public KinesisProducerConfiguration setRecordTtl(long val)
This setting is useful if your application cannot or does not wish to tolerate late records. Records will still incur network latency after they leave the KPL, so take that into consideration when choosing a value for this setting.
If you do not wish to lose records and prefer to retry indefinitely, set record_ttl to a large value like INT_MAX. This has the potential to cause head-of-line blocking if network issues or throttling occur. You can respond to such situations by using the metrics reporting functions of the KPL. You may also set fail_if_throttled to true to prevent automatic retries in case of throttling.
Default: 30000
Minimum: 100
Maximum (inclusive): 9223372036854775807
public KinesisProducerConfiguration setRegion(String val)
If you do not specify the region and are running in EC2, the library will use the region the instance is in.
The region is also used to sign requests.
Expected pattern: ^([a-z]+-([a-z]+-)?[a-z]+-[0-9])?$
public KinesisProducerConfiguration setRequestTimeout(long val)
Note that a timed-out request may actually succeed at the backend. Retrying then leads to duplicates. Setting the timeout too low will therefore increase the probability of duplicates.
Default: 6000
Minimum: 100
Maximum (inclusive): 600000
public KinesisProducerConfiguration setTempDirectory(String val)
If not specified, defaults to /tmp in Unix. (Windows TBD)
public KinesisProducerConfiguration setVerifyCertificate(boolean val)
Default: true
public KinesisProducerConfiguration setThreadingModel(KinesisProducerConfiguration.ThreadingModel threadingModel)
getThreadingModel()
for more informationthreadingModel
- the threading model to usepublic KinesisProducerConfiguration setThreadingModel(String threadingModel)
See getThreadingModel()
for more information
Valid inputs: PER_REQUEST - Tells the native process to create a thread per request. POOLED - Tells the native process to use a thread pool. The size of the pool can be controlled by ThreadPoolSize.
threadingModel
- the String representation of the threading model to use.public KinesisProducerConfiguration setThreadPoolSize(int threadPoolSize)
getThreadPoolSize()
for more informationthreadPoolSize
- the maximum number of threads that the thread pool can use.IllegalArgumentException
- if threadPoolSize is less than 0protected com.amazonaws.services.kinesis.producer.protobuf.Messages.Message toProtobufMessage()
Copyright © 2019. All rights reserved.