Class RpcQosOptions
- java.lang.Object
-
- org.apache.beam.sdk.io.gcp.firestore.RpcQosOptions
-
- All Implemented Interfaces:
java.io.Serializable
,org.apache.beam.sdk.transforms.display.HasDisplayData
@Immutable @ThreadSafe public final class RpcQosOptions extends java.lang.Object implements java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
Quality of Service manager options for Firestore RPCs.Every RPC which is sent to Cloud Firestore is subject to QoS considerations. Successful, failed, attempted requests are all tracked and directly drive the determination of when to attempt an RPC.
Configuration of options can be accomplished by passing an instances of
RpcQosOptions
to thewithRpcQosOptions
method of eachBuilder
available inFirestoreV1
.A new instance of
RpcQosOptions.Builder
can be created vianewBuilder()
. A default instance ofRpcQosOptions
can be created viadefaultOptions()
.- See Also:
FirestoreV1
,FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
,FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
,FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
,FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
,FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
,FirestoreV1.Transform.Builder.withRpcQosOptions(RpcQosOptions)
, Standard limits, Designing for scale, Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
RpcQosOptions.Builder
Mutable Builder class for creating instances ofRpcQosOptions
.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static RpcQosOptions
defaultOptions()
Factory method to return a new instance ofRpcQosOptions
with all default values.boolean
equals(@Nullable java.lang.Object o)
int
getBatchInitialCount()
The initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.long
getBatchMaxBytes()
The maximum number of bytes to include in a batch.int
getBatchMaxCount()
The maximum number of writes to include in a batch.org.joda.time.Duration
getBatchTargetLatency()
Target latency for batch requests.int
getHintMaxNumWorkers()
A hint to the QoS system for the intended max number of workers for a pipeline.org.joda.time.Duration
getInitialBackoff()
The initial backoff duration to be used before retrying a request for the first time.int
getMaxAttempts()
The maximum number of times a request will be attempted for a complete successful result.double
getOverloadRatio()
The target ratio between requests sent and successful requests.org.joda.time.Duration
getSamplePeriod()
The length of time sampled request data will be retained.org.joda.time.Duration
getSamplePeriodBucketSize()
The size of buckets within the specifiedsamplePeriod
.org.joda.time.Duration
getThrottleDuration()
The amount of time an attempt will be throttled if deemed necessary based on previous success rate.int
hashCode()
boolean
isShouldReportDiagnosticMetrics()
Whether additional diagnostic metrics should be reported for a Transform.static RpcQosOptions.Builder
newBuilder()
Factory method to return a new instance ofRpcQosOptions.Builder
with all values set to their initial default values.void
populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
RpcQosOptions.Builder
toBuilder()
Create a newRpcQosOptions.Builder
initialized with the values from this instance.java.lang.String
toString()
-
-
-
Method Detail
-
populateDisplayData
public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
- Specified by:
populateDisplayData
in interfaceorg.apache.beam.sdk.transforms.display.HasDisplayData
-
getMaxAttempts
public int getMaxAttempts()
The maximum number of times a request will be attempted for a complete successful result.For a stream based response, the full stream read must complete within the specified number of attempts. Restarting a stream will count as a new attempt.
Default Value: 5
-
getInitialBackoff
public org.joda.time.Duration getInitialBackoff()
The initial backoff duration to be used before retrying a request for the first time.Default Value: 5 sec
-
getSamplePeriod
public org.joda.time.Duration getSamplePeriod()
The length of time sampled request data will be retained.Default Value: 2 min
-
getSamplePeriodBucketSize
public org.joda.time.Duration getSamplePeriodBucketSize()
The size of buckets within the specifiedsamplePeriod
.Default Value: 10 sec
-
getOverloadRatio
public double getOverloadRatio()
The target ratio between requests sent and successful requests. This is "K" in the formula in SRE Book - Handling OverloadDefault Value: 1.05
-
getThrottleDuration
public org.joda.time.Duration getThrottleDuration()
The amount of time an attempt will be throttled if deemed necessary based on previous success rate.Default value: 5 sec
-
getBatchInitialCount
public int getBatchInitialCount()
The initial size of a batch; used in the absence of the QoS system having significant data to determine a better batch size.Default Value: 20
-
getBatchMaxCount
public int getBatchMaxCount()
The maximum number of writes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of writes per request may be lower if we reachbatchMaxBytes
.Default Value: 500
-
getBatchMaxBytes
public long getBatchMaxBytes()
The maximum number of bytes to include in a batch. (Used in the absence of the QoS system having significant data to determine a better value). The actual number of bytes per request may be lower ifbatchMaxCount
is reached first.Default Value: 9.5 MiB
-
getBatchTargetLatency
public org.joda.time.Duration getBatchTargetLatency()
Target latency for batch requests. It aims for a target response time per RPC: the response times for previous RPCs and the number of writes contained in them, calculates a rolling average time-per-write, and chooses the number of writes for future requests to hit the target time.Default Value: 5 sec
-
getHintMaxNumWorkers
public int getHintMaxNumWorkers()
A hint to the QoS system for the intended max number of workers for a pipeline. The provided value can be used to try to scale calculations to values appropriate for the pipeline as a whole.If you are running your pipeline on Cloud Dataflow this parameter should be set to the same value as
maxNumWorkers
Default Value: 500
-
isShouldReportDiagnosticMetrics
public boolean isShouldReportDiagnosticMetrics()
Whether additional diagnostic metrics should be reported for a Transform.If true, additional detailed diagnostic metrics will be reported for the RPC QoS subsystem.
This parameter should be used with care as it will output ~40 additional custom metrics which will count toward any possible pipeline metrics limits (For example Dataflow Custom Metrics limits and Cloud Monitoring custom metrics quota)
Default Value:
false
-
toBuilder
public RpcQosOptions.Builder toBuilder()
Create a newRpcQosOptions.Builder
initialized with the values from this instance.- Returns:
- a new
RpcQosOptions.Builder
initialized with the values from this instance.
-
equals
public boolean equals(@Nullable java.lang.Object o)
- Overrides:
equals
in classjava.lang.Object
-
hashCode
public int hashCode()
- Overrides:
hashCode
in classjava.lang.Object
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
defaultOptions
public static RpcQosOptions defaultOptions()
Factory method to return a new instance ofRpcQosOptions
with all default values.- Returns:
- New instance of
RpcQosOptions
with all default values - See Also:
newBuilder()
-
newBuilder
public static RpcQosOptions.Builder newBuilder()
Factory method to return a new instance ofRpcQosOptions.Builder
with all values set to their initial default values.- Returns:
- New instance of
RpcQosOptions.Builder
with all values set to their initial default values - See Also:
defaultOptions()
-
-